Working with Tabular Data

In this section of the tutorial, we will show how to work with tabular data using the Blackfynn Python client. We recommend you that, in order to get the most out of this tutorial, you look into the Working with the data catalog tutorial first.

Through this tutorial, we will show some examples that will show some of the methods that the Blackfynn python client offers for working with tabular data. You can find the details about the supported file formants in the file formats section.

We will be using the demographics and disease severity information file for the Gait in Parkinson’s Disease database. We have included this CSV file in the tutorial’s data directory. However, this dataset is publicly available and can also be obtained from the Physionet website.

Tabular Data Basics

We will start by uploading the tabular data to the platform and downloading it in the standard format in order to demonstrate some of the download parameters.

1
2
3
4
5
6
7
8
 # import blackfynn
 from blackfynn import Blackfynn

 # create a client instance
 bf = Blackfynn()

 # create a dataset in the platform to save our tabular data and get dataset object
 ds = bf.create_dataset('Tabular Dataset')
1
2
3
4
5
 # upload file to platform
 ds.upload('example_data/gait.csv');

 for item in ds:
     print "Type:", item.type, "|", "Name:", item.name, "|" , "ID:", item.id

Note

If you are uploading large files, you might not see all of your packages right away. You might have to wait for a few seconds. To check if your package is ready, you can get the package’s state through the state attribute of the package’s object. If the package is done uploading and ready, pkg.state should return READY.

1
 Type: Tabular | Name: gait | ID: N:package:35716bd1-dde0-4c09-b7ab-04a63b0ac29f

We see that our package has been successfully uploaded to the Blackfynn platform and that it has been assigned the type Tabular.

Downloading Tabular Data

So far, we have uploaded a CSV file and created a Tabular package in the Balckfynn platform. We will now demonstrate partial and full download of the data through the get_data() method.

First, we will obtain the entire dataset.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
 # get the package object through its ID
 tb = bf.get('N:package:35716bd1-dde0-4c09-b7ab-04a63b0ac29f')

 # get all the data
 data = tb.get_data()

 # print rows and column information for the data
 print("Data has {} rows and {} columns").format(len(data.index),len(data.columns))
 print("First Index: {}").format(data.index[0])
 print("Last Index: {}").format(data.index[len(data.index)-1])

 # print the column names for the data
 print "\nColumns in the data:"
 print " | ".join(data.columns)

 data
Data has 166 rows and 16 columns
First Index: 0
Last Index: 165

Columns in the data:
id | subject_type | speed_01 | speed_10 | reference | v_lastmodified_epoch | v_status | v_uuid | gender | age | height | weight | hoehnyahr | updrs | updrsm | tuag
data
  id subject_type speed_01 speed_10 reference v_lastmodified_epoch v_status v_uuid gender age height weight hoehnyahr updrs updrsm tuag
0 GaPt03 Parkinsons Patient, Gait study   0.778 PMID:16176368 1460000000.0 E 69cabe6a-8b75-5fa1-ab43-5a15ab118b26 female 82 1.45 50.0 3.0 20.0 10.0 36.34
1 GaPt04 Parkinsons Patient, Gait study 0.642 0.818 PMID:16176368 1460000000.0 E c57a9ac1-6388-5b38-9f24-94ffa725be36 male 68 1.71   2.5 25.0 8.0 11.0
2 GaPt05 Parkinsons Patient, Gait study 0.908 0.614 PMID:16176368 1460000000.0 E 2b0b804d-5a74-5fc8-a641-f9f2a9d5b578 female 82 1.53 51.0 2.5 24.0 5.0 14.5
3 GaPt06 Parkinsons Patient, Gait study 0.848 0.937 PMID:16176368 1460000000.0 E d665e2a2-3a77-57c6-a091-dafda73929c6 male 72 1.7 82.0 2.0 16.0 13.0 10.47
4 GaPt07 Parkinsons Patient, Gait study 0.677 0.579 PMID:16176368 1460000000.0 E 78a1315f-1a0e-55fa-8be6-279b561fddaa female 53 1.67 54.0 3.0 44.0 22.0 18.34
5 GaPt08 Parkinsons Patient, Gait study 1.046 0.228 PMID:16176368 1460000000.0 E 9e6ee29e-b072-57c4-aaf5-e3b63e9e194f female 68 1.63 57.0 2.0 15.0 8.0 10.11
6 GaPt09 Parkinsons Patient, Gait study 0.894 1.253 PMID:16176368 1460000000.0 E d3cc6b8e-9406-5c09-823a-2e86435712e8 male 69 1.6 68.0 3.0 34.0 17.0 12.7
7 GaPt12 Parkinsons Patient, Gait study 1.261 1.133 PMID:16176368 1460000000.0 E 15de656e-e3b6-5c65-af2a-3041056b5b9a female 59 1.63 67.0 2.0 25.0 7.0 8.37
8 GaPt13 Parkinsons Patient, Gait study 0.726 0.798 PMID:16176368 1460000000.0 E e47caedb-10b5-5de3-a4c7-e14eb944ef70 male 70 1.68 53.0 2.0 38.0 21.0 15.51
9 GaPt14 Parkinsons Patient, Gait study 1.369 0.973 PMID:16176368 1460000000.0 E d0d35e6d-c43b-59e7-b30b-a2b1ca9dafc9 male 56 1.95 105.0 2.0 29.0 19.0  
10 GaPt15 Parkinsons Patient, Gait study 0.948 0.899 PMID:16176368 1460000000.0 E 57db5383-13e7-5c31-94f0-5dec5b3bbab7 male 81   80.0 2.0 33.0 20.0 10.32
11 JuPt24 Parkinsons Patient, Gait study 0.918   PMID:17953624 1460000000.0 E ea354138-15ea-5d7b-9a66-e9629e52bb00 male 68 163.0 75.0 2.5 40.0 23.0 10.66
12 GaPt16 Parkinsons Patient, Gait study 1.048 1.129 PMID:16176368 1460000000.0 E ad5c99cc-4ceb-58c9-a92d-f8b29577fffe male 79 1.7 72.0 2.0 18.0 10.0 8.74
13 GaPt17 Parkinsons Patient, Gait study 0.731 0.713 PMID:16176368 1460000000.0 E 1d0c2006-f472-5e74-a99e-221ac41cd915 male 71 1.82 85.0 3.0 44.0 27.0 14.81
14 GaPt18 Parkinsons Patient, Gait study 0.889 1.101 PMID:16176368 1460000000.0 E 35d15970-f97a-5ed9-8a83-403851f21e63 female 76 1.52 74.0 2.0 29.0 20.0 12.72
15 GaPt19 Parkinsons Patient, Gait study 1.124 0.87 PMID:16176368 1460000000.0 E e8f88d35-df39-52a0-870a-0d444e6018c4 female 76   55.0 2.0 30.0 22.0 13.2
16 GaPt20 Parkinsons Patient, Gait study 0.722   PMID:16176368 1460000000.0 E 3562d198-56e0-5b0b-b678-ba39d2c2790c male 81 1.65 75.0 2.0 63.0 36.0 13.71
17 GaPt21 Parkinsons Patient, Gait study 1.166   PMID:16176368 1460000000.0 E 8f05a553-7d87-5111-b9f8-e9fb2f127663 male 80   63.0 3.0 46.0 27.0 14.06
18 GaPt22 Parkinsons Patient, Gait study 0.802   PMID:16176368 1460000000.0 E d00cbdb7-bcf2-56dc-ae96-6a511a40393d male 78 1.78 65.0 2.0 49.0 29.0  
19 GaPt23 Parkinsons Patient, Gait study 0.36   PMID:16176368 1460000000.0 E 7e21b1d6-28e3-5392-b3c2-3963d2513e9d female 71 1.6   3.0 51.0 25.0 25.01
20 GaPt24 Parkinsons Patient, Gait study 1.255   PMID:16176368 1460000000.0 E a101da76-087f-508f-828a-534e8ea2ac39 male 68 1.72 73.0 2.5 42.0 15.0 11.42
21 GaPt25 Parkinsons Patient, Gait study 1.128   PMID:16176368 1460000000.0 E 3262c86a-8fb1-58e3-a775-390037e01710 male 81 1.76 90.0 2.5 31.0 18.0 15.22
22 GaPt26 Parkinsons Patient, Gait study 1.244   PMID:16176368 1460000000.0 E bba41392-f156-5711-8448-34bdc77d3de8 female 78 1.52 60.0 2.5 24.0 5.0 7.27
23 GaPt27 Parkinsons Patient, Gait study 1.423   PMID:16176368 1460000000.0 E 47e0d47c-a884-5a6a-af80-1c8786131e3e male 72 1.8 95.0 2.0 21.0 10.0 7.88
24 GaPt28 Parkinsons Patient, Gait study 0.987   PMID:16176368 1460000000.0 E 468a9868-6f93-5329-ac50-0d3a95254a11 male 61 1.79 101.0 2.5 54.0 29.0 13.02
25 GaPt29 Parkinsons Patient, Gait study 1.092   PMID:16176368 1460000000.0 E 0c7e8115-e496-5ab1-b723-2bfbedd0e2b7 male 68 1.63 80.0 2.0 27.0 16.0 10.16
26 GaPt30 Parkinsons Patient, Gait study 1.064   PMID:16176368 1460000000.0 E 27656eaf-cfa0-5b31-8105-124d60e193f5 male 69 1.78 93.0 2.0 20.0 12.0 9.91
27 GaPt31 Parkinsons Patient, Gait study 0.876   PMID:16176368 1460000000.0 E 0b5bf70a-6720-58c2-965b-cdc061f0d991 male 67 1.76 90.0 2.5 27.0 13.0 12.6
28 GaPt32 Parkinsons Patient, Gait study 1.242   PMID:16176368 1460000000.0 E e7c80672-61fa-599b-8748-8ee97a511edd male 63 1.69 75.0 2.0 33.0 24.0 11.22
29 GaPt33 Parkinsons Patient, Gait study 0.825   PMID:16176368 1460000000.0 E c8c96583-df0b-5050-b2a3-efcc20f8d49e male 63 1.86 80.0 2.5 42.0 31.0 11.97
30 JuPt01 Parkinsons Patient, Gait study 1.013   PMID:17953624 1460000000.0 E 7e977d95-c386-5368-8db9-7f6427df3444 male 77 183.0 85.0 2.0 15.0 11.0 15.5
31 JuPt02 Parkinsons Patient, Gait study 0.906   PMID:17953624 1460000000.0 E 24a1695e-4f57-5dc7-93fe-46a64ce534d5 female 72 160.0 68.0 2.5 32.0 21.0 12.47
32 JuPt03 Parkinsons Patient, Gait study 0.993   PMID:17953624 1460000000.0 E 926ddad8-685f-5c2b-9127-e79750737d65 female 74 164.0 65.0 2.5 14.0 10.0 9.59
33 JuPt04 Parkinsons Patient, Gait study 0.807   PMID:17953624 1460000000.0 E 8d635876-d982-5340-8eec-619fa0d9a337 male 70 173.0 80.0 2.5 34.0 19.0 13.75
34 JuPt05 Parkinsons Patient, Gait study 1.234   PMID:17953624 1460000000.0 E 271b2e11-09c8-5e74-a3f4-8b74520048be male 74 170.0 61.0 2.0 15.0 14.0 11.16
35 JuPt06 Parkinsons Patient, Gait study 0.832   PMID:17953624 1460000000.0 E abe91d77-8789-5bf3-88a5-b5b16ffe8648 male 78 175.0 82.0 2.5 21.0 12.0 16.29
36 JuPt07 Parkinsons Patient, Gait study 1.185   PMID:17953624 1460000000.0 E 628b6bfe-aed9-5e02-b1d2-3ea928187223 female 51 160.0 65.0 3.0 24.0 14.0 9.65
37 JuPt08 Parkinsons Patient, Gait study 1.246   PMID:17953624 1460000000.0 E c81e5bd3-42f3-53df-b338-1644e0b7c222 female 54 159.0 70.0 2.0 26.0 16.0 8.5
38 JuPt09 Parkinsons Patient, Gait study 1.146   PMID:17953624 1460000000.0 E 086db299-2ecf-5399-beec-bbefa33a7bef male 61 170.0 82.0 2.5 27.0 10.0 10.81
39 JuPt10 Parkinsons Patient, Gait study 1.218   PMID:17953624 1460000000.0 E 9d1bddc5-93aa-59d5-acd2-4ea311294dc7 male 69 170.0 80.0 3.0 28.0 20.0 10.96
40 JuPt11 Parkinsons Patient, Gait study 0.785   PMID:17953624 1460000000.0 E 2be6199c-0f55-5631-97bd-3232215f6f3a male 74 175.0 80.0 2.0 24.0 16.0 12.81
41 JuPt12 Parkinsons Patient, Gait study 1.052   PMID:17953624 1460000000.0 E 12ddc9db-8f21-5456-91d7-a9dcb3bca815 female 58 156.0 56.0 2.5 21.0 14.0 9.85
42 JuPt13 Parkinsons Patient, Gait study 0.664   PMID:17953624 1460000000.0 E ccaeaac0-f7ea-57cb-9ca4-28abf8c4edc2 male 80 164.0 60.0 3.0 39.0 23.0 16.09
43 JuPt14 Parkinsons Patient, Gait study 0.906   PMID:17953624 1460000000.0 E 5827c5d1-61a3-541c-a654-712c750cced2 female 67 158.0 63.0 2.0 13.0 8.0 10.5
44 JuPt15 Parkinsons Patient, Gait study 1.099   PMID:17953624 1460000000.0 E 272c3f33-f9c2-547a-b755-3b95ee25d2b9 female 64 162.0 66.0 2.5 18.0 10.0 10.0
45 JuPt16 Parkinsons Patient, Gait study 0.755   PMID:17953624 1460000000.0 E 750d7ed6-be03-5ab9-966f-5d3631658857 female 64 150.0 63.0 2.0 19.0 16.0 12.22
46 JuPt17 Parkinsons Patient, Gait study 1.112   PMID:17953624 1460000000.0 E 1f2b6d9a-94ce-5dec-a2a0-29c56acf0510 male 64 178.0 75.0 2.5 24.0 18.0 10.03
47 JuPt18 Parkinsons Patient, Gait study 0.413   PMID:17953624 1460000000.0 E 7ced57e0-f892-5a62-a767-363321e467cf male 82 160.0 65.0 3.0 33.0 16.0 25.25
48 JuPt19 Parkinsons Patient, Gait study 1.183   PMID:17953624 1460000000.0 E 7bf3a123-a4c2-5b6a-a359-b901b2e7b881 female 68 160.0 60.0 2.0 17.0 13.0 10.0
49 JuPt20 Parkinsons Patient, Gait study 1.212   PMID:17953624 1460000000.0 E 220d68bc-896d-5ca6-a51d-3b8045c96c6a male 64 168.0 80.0 2.0 33.0 24.0 11.22
50 JuPt21 Parkinsons Patient, Gait study 0.616   PMID:17953624 1460000000.0 E 1fa368a9-62cb-5aeb-86c8-8638129ae9ab male 84 173.0 75.0 2.5 23.0 18.0 15.31
51 JuPt22 Parkinsons Patient, Gait study 1.178   PMID:17953624 1460000000.0 E 9f0ac01b-fff1-5294-814d-21023ac556eb male 69 170.0 70.0 2.5 38.0 18.0 8.81
52 JuPt23 Parkinsons Patient, Gait study 1.247   PMID:17953624 1460000000.0 E fdaa3747-43ed-5b18-ad90-4ad77c7709a4 male 68 160.0 69.0 2.0 19.0 16.0 7.53
53 JuPt25 Parkinsons Patient, Gait study 1.087   PMID:17953624 1460000000.0 E 8a4d907b-041f-53d8-88a3-cb1e4b0e317d male 45 174.0 68.0 2.0 23.0 14.0 10.63
54 JuPt26 Parkinsons Patient, Gait study 0.94   PMID:17953624 1460000000.0 E 7195db5a-e3bf-506a-87ef-529239baf08a female 66 167.0 84.0 2.0 18.0 11.0 12.47
55 JuPt27 Parkinsons Patient, Gait study 1.135   PMID:17953624 1460000000.0 E 3ba1df19-4980-542d-9a2f-eedf0b2aef39 female 58 170.0 65.0 2.0 16.0 13.0 12.67
56 JuPt28 Parkinsons Patient, Gait study 1.118   PMID:17953624 1460000000.0 E 5a6976fb-affe-56df-b1ab-c77d83005b55 female 57 175.0 65.0 2.5 43.0 25.0 11.43
57 JuPt29 Parkinsons Patient, Gait study 1.044   PMID:17953624 1460000000.0 E 14e6a8b5-7e3b-56ff-b69b-74156240c6c8 female 69 160.0 61.0 2.5 25.0 15.0 10.49
58 SiPt02 Parkinsons Patient, Gait study 0.77   PMID:15929090 1460000000.0 E 186590a5-f5d8-5a2c-8e31-2728e7c34122 female 77 1.58 67.0 2.5      
59 SiPt04 Parkinsons Patient, Gait study 1.147   PMID:15929090 1460000000.0 E 59674725-8a47-5bbb-b516-6517993d3d2a male 73 1.7 70.0 2.0 27.0 19.0 14.26
60 SiPt05 Parkinsons Patient, Gait study 1.276   PMID:15929090 1460000000.0 E a772fd32-3a83-58e5-9727-496fe448e559 male 56 1.72 76.0 2.5 50.0 32.0 10.38
61 SiPt07 Parkinsons Patient, Gait study 1.088   PMID:15929090 1460000000.0 E af4d21a7-c49c-567d-9716-ab1575c0f176 female 54 1.66 65.0 2.0     10.99
62 SiPt08 Parkinsons Patient, Gait study 1.187   PMID:15929090 1460000000.0 E de190b07-8473-59f0-ab74-696156efad25 male 67 1.75 100.0 2.0 56.0 33.0 10.83
63 SiPt09 Parkinsons Patient, Gait study 1.352   PMID:15929090 1460000000.0 E f88fb11c-3aaa-56c6-9388-7477ff4abac1 male 63 1.74 80.0 2.0 21.0 12.0 7.94
64 SiPt10 Parkinsons Patient, Gait study 0.935   PMID:15929090 1460000000.0 E dc1757ae-49d1-5f34-ad43-98429c4de522 female 77 1.6 60.0 2.0 41.0 27.0 14.41
65 SiPt12 Parkinsons Patient, Gait study 0.988   PMID:15929090 1460000000.0 E c2a4c483-bf4c-5907-a277-5eff387db498 female 64 1.67 55.0 2.0 44.0 34.0 10.28
66 SiPt13 Parkinsons Patient, Gait study 1.107   PMID:15929090 1460000000.0 E 08067471-f457-55ec-92a3-ba984ef95de3 male 52 1.65 75.0 2.0 42.0 27.0 10.38
67 SiPt14 Parkinsons Patient, Gait study 1.218   PMID:15929090 1460000000.0 E 7b717515-75d2-544d-b6a4-ee71765b843e male 60 1.65 65.0 2.5 46.0 32.0 8.13
68 SiPt15 Parkinsons Patient, Gait study 1.031   PMID:15929090 1460000000.0 E 5ee0a238-e7dc-5e20-b840-2f6b5637cfb9 male 62 1.65 82.0 2.0 32.0 19.0 10.89
69 SiPt16 Parkinsons Patient, Gait study 1.139   PMID:15929090 1460000000.0 E 435933f2-71ec-5361-8fe2-25276e594041 male 50 1.73 69.0 2.0 41.0 26.0 9.09
70 SiPt17 Parkinsons Patient, Gait study 1.215   PMID:15929090 1460000000.0 E 422bbe23-c7c5-50ee-b0b9-c4392e17b7a5 male 64 1.65 80.0 2.0 36.0 24.0 11.56
71 SiPt18 Parkinsons Patient, Gait study 1.117   PMID:15929090 1460000000.0 E 2a301c34-4b8c-5871-830f-aaeb7fb5a2b6 male 57 1.6 88.0 2.0 48.0 30.0 14.39
72 SiPt19 Parkinsons Patient, Gait study 1.298   PMID:15929090 1460000000.0 E ccf485d6-2eca-56aa-8c10-5d7afebe5900 male 53 1.8 76.0 2.0 40.0 29.0 11.07
73 SiPt20 Parkinsons Patient, Gait study 1.032   PMID:15929090 1460000000.0 E 70d2f6f2-7125-59c2-b6bf-8e6bba69b798 male 59 1.76 77.0 2.0 26.0 18.0 13.16
74 SiPt21 Parkinsons Patient, Gait study 1.151   PMID:15929090 1460000000.0 E 82352bca-82d7-5599-9e3a-5257595f0c3c male 58 1.67 66.0 2.0 24.0 15.0 10.48
75 SiPt22 Parkinsons Patient, Gait study 1.021   PMID:15929090 1460000000.0 E 2a4ec68c-e380-5a6f-b17f-5e948392c9f6 male 71 1.68 67.0 2.0 31.0 19.0 11.99
76 SiPt23 Parkinsons Patient, Gait study 1.088   PMID:15929090 1460000000.0 E 74309506-ce01-57e7-ae0f-db5394fcd7ef female 50 1.54 82.0 2.0 26.0 20.0 11.08
77 SiPt24 Parkinsons Patient, Gait study 1.192   PMID:15929090 1460000000.0 E fafcb6f6-b46c-5968-aa51-3fa0809b1371 female 36 1.64 52.0 2.0 24.0 17.0 9.53
78 SiPt25 Parkinsons Patient, Gait study 1.167   PMID:15929090 1460000000.0 E f98f7048-a555-5e61-be0a-47d60724c61e male 63 1.68 80.0 2.0 33.0 24.0 11.22
79 SiPt27 Parkinsons Patient, Gait study 1.259   PMID:15929090 1460000000.0 E b44fb6f4-9ffc-53ef-9b02-f183a7716b8c male 50 1.72 75.0 2.0 46.0 26.0 11.32
80 SiPt28 Parkinsons Patient, Gait study 1.204   PMID:15929090 1460000000.0 E c834244d-a286-55b8-b05d-f08748a76eac female 70 1.58 72.0 2.5 33.0 18.0 10.98
81 SiPt29 Parkinsons Patient, Gait study 0.856   PMID:15929090 1460000000.0 E 99350401-b532-54dd-b707-c114c2e19b0e female 72 1.73 85.0 2.5 70.0 44.0 15.51
82 SiPt30 Parkinsons Patient, Gait study 1.162   PMID:15929090 1460000000.0 E f14e974d-35e3-532c-8a8a-6c4001950979 male 60 1.64 89.0 2.0 15.0 12.0 11.55
83 SiPt31 Parkinsons Patient, Gait study 0.903   PMID:15929090 1460000000.0 E 37bb8bbd-7f94-5d70-92af-98ece5d84a60 female 56 1.6 74.0 2.0 39.0 24.0 9.97
84 SiPt32 Parkinsons Patient, Gait study 1.12   PMID:15929090 1460000000.0 E 9a393e62-4275-5c49-a971-452d47fd01e5 male 64 1.66 75.0 2.0 32.0 23.0 12.83
85 SiPt33 Parkinsons Patient, Gait study 1.08   PMID:15929090 1460000000.0 E a151b917-7a39-5fc9-acbb-7ebc62193086 male 67 1.76 74.0 2.0 28.0 15.0 10.57
86 SiPt34 Parkinsons Patient, Gait study 1.386   PMID:15929090 1460000000.0 E 6cd8f699-d693-56ad-bfe1-2ced8572a3c3 male 71 1.8 68.0 2.0 37.0 22.0 11.63
87 SiPt35 Parkinsons Patient, Gait study 1.28   PMID:15929090 1460000000.0 E b7db79ef-b742-5365-9883-7659f093dc84 male 57 1.74 95.0 2.0 42.0 29.0 8.77
88 SiPt36 Parkinsons Patient, Gait study 0.97   PMID:15929090 1460000000.0 E dbba4d91-1f99-5479-8c0f-9c4303bf7ffa female 53 1.58 62.0 2.0 52.0 32.0 11.27
89 SiPt37 Parkinsons Patient, Gait study 1.01   PMID:15929090 1460000000.0 E 76450708-2bd5-501e-bc9e-310319a14926 female 66 1.7 62.0 2.5 27.0 21.0 7.56
90 SiPt38 Parkinsons Patient, Gait study 1.07   PMID:15929090 1460000000.0 E c2c05c22-ed78-55d9-8079-83343a666c48 female 65 1.59 60.0 2.0 22.0 14.0 10.13
91 SiPt39 Parkinsons Patient, Gait study 0.88   PMID:15929090 1460000000.0 E b35581f9-668f-5a00-855b-88f5ef21047b female 69 1.68 53.0 2.0 33.0 20.0 13.97
92 SiPt40 Parkinsons Patient, Gait study 1.07   PMID:15929090 1460000000.0 E 5d40dc30-c6e1-573f-ae61-4a86bd134657 male 69 1.6 81.0 2.5 37.0 24.0 9.7
93 GaCo01 Control Patient, Gait study 1.075   PMID:16176368 1460000000.0 E e20e025b-ce75-5ed9-bf80-76be8eca6e39 male 66 1.8 83.0 0.0 0.0 0.0  
94 GaCo02 Control Patient, Gait study 1.04   PMID:16176368 1460000000.0 E bf8eee54-032a-5e4b-bcfe-6766b8570977 male 74 1.74 70.0 0.0 1.0 1.0  
95 GaCo03 Control Patient, Gait study 1.051   PMID:16176368 1460000000.0 E e8fb8f97-8e7c-5bb9-86aa-f251cf33cc33 male 69 1.8 101.0 0.0 0.0 0.0  
96 GaCo04 Control Patient, Gait study 1.175   PMID:16176368 1460000000.0 E 788b704f-1898-5f65-85fe-98c0a3f95abc male 86 1.66 65.0 0.0 3.0 3.0 9.57
97 GaCo05 Control Patient, Gait study 0.92   PMID:16176368 1460000000.0 E 5a96f84d-a696-5889-93b2-9f2b4a90e098 female 75 1.5 88.0 0.0 0.0 0.0  
98 GaCo06 Control Patient, Gait study 1.121   PMID:16176368 1460000000.0 E 5b99d70b-24e0-596a-8550-84a0871493c9 male 82 1.68 82.0 0.0 1.0 1.0 9.7
99 GaCo07 Control Patient, Gait study 1.282   PMID:16176368 1460000000.0 E 91f62be5-6cf1-5424-b06b-76e5f323921d male 79 1.68 95.0 0.0 1.0 1.0  
100 GaCo08 Control Patient, Gait study 0.975   PMID:16176368 1460000000.0 E 1706e642-ce60-5af0-97c0-d0294fdf1cd9 female 78 1.58 65.0 0.0 1.0 1.0  
101 GaCo09 Control Patient, Gait study 1.249   PMID:16176368 1460000000.0 E 6683e6b4-b9fb-51fc-893d-362a2a6da22e female 78 1.63 64.0 0.0 1.0 1.0  
102 GaCo10 Control Patient, Gait study 1.164   PMID:16176368 1460000000.0 E ba24fa41-42d9-5803-8770-4ef5c3c888e1 male 69 1.72 70.0 0.0 0.0 0.0 10.12
103 GaCo11 Control Patient, Gait study 1.515   PMID:16176368 1460000000.0 E 0915b132-1790-5901-886b-2d12169186d2 male 70 1.75 80.0 0.0 3.0 3.0 7.16
104 GaCo12 Control Patient, Gait study 1.389   PMID:16176368 1460000000.0 E 080a56cb-8a3f-5ba8-b7e5-045f11b8e740 female 65 1.72 60.0 0.0 1.0 1.0 7.49
105 GaCo13 Control Patient, Gait study 1.211   PMID:16176368 1460000000.0 E 1a9fdfb0-53ea-5271-bf3e-262cdcdd83d6 female 67 1.58 72.0 0.0 0.0 0.0 8.1
106 GaCo14 Control Patient, Gait study 1.298 1.248 PMID:16176368 1460000000.0 E 95c3d751-8e41-575d-9d5c-3caf3e4008c4 female 68 1.7 47.0 0.0 2.0 2.0 8.11
107 GaCo15 Control Patient, Gait study 1.344 1.338 PMID:16176368 1460000000.0 E a8d7e609-e349-5c45-9f79-7a72997ab5cf female 67 1.7 70.0 0.0 0.0 0.0 8.37
108 GaCo16 Control Patient, Gait study 1.346 1.234 PMID:16176368 1460000000.0 E 9bdf1e12-c23b-5b71-b312-baf56d09ab6f female 63 1.55 58.0 0.0 0.0 0.0 8.84
109 GaCo17 Control Patient, Gait study 1.415 1.174 PMID:16176368 1460000000.0 E b6fcf3f7-de58-5f05-adc3-3fe99080ad65 male 67 1.62 58.0 0.0 0.0 0.0 7.19
110 GaCo22 Control Patient, Gait study 1.542 1.532 PMID:16176368 1460000000.0 E 2414013b-8b88-51e6-87c1-876b7ac3000f male 65 1.72 72.0 0.0 0.0 0.0  
111 JuCo01 Control Patient, Gait study 1.089   PMID:17953624 1460000000.0 E 57e5097b-6691-595e-900c-0b20eb620b7e male 78 167.0 75.0   0.0 0.0 10.19
112 JuCo02 Control Patient, Gait study 1.329   PMID:17953624 1460000000.0 E b4be54ff-d3fa-5593-87c4-27c83e4fd787 male 71 175.0 80.0   3.0 3.0 7.16
113 JuCo03 Control Patient, Gait study 1.322   PMID:17953624 1460000000.0 E 36aa258f-be89-59b7-ac70-1eae9731ecd4 female 66 172.0 60.0   1.0 1.0 7.49
114 JuCo04 Control Patient, Gait study 1.239   PMID:17953624 1460000000.0 E fd21ce96-1cdc-5f8f-849e-83b71a28bea8 female 68 170.0 80.0   1.0 1.0 9.27
115 JuCo05 Control Patient, Gait study 1.248   PMID:17953624 1460000000.0 E a1df9893-f61c-54c7-a4b5-6e02c264f94e female 55 159.0 52.0   0.0 0.0 9.09
116 JuCo06 Control Patient, Gait study 0.954   PMID:17953624 1460000000.0 E 5cb8bbd4-e24f-5038-91e8-17f47536c3c1 female 74 155.0 74.0   0.0 0.0 9.91
117 JuCo07 Control Patient, Gait study 1.332   PMID:17953624 1460000000.0 E f615ebbe-1ce5-5119-bb47-69b888d19b8d female 62 160.0 58.0   2.0 2.0 8.31
118 JuCo08 Control Patient, Gait study 1.286   PMID:17953624 1460000000.0 E 6f8ae96d-04c7-5fdc-8c6b-9c700ae3ff71 female 77 170.0 65.0   0.0 0.0 9.22
119 Juc010 Control Patient, Gait study 1.375   PMID:17953624 1460000000.0 E 0b50e466-7506-58ba-b7d9-8b9b750ac1e9 female 62 167.0 78.0   0.0 0.0 8.82
120 JuCo09 Control Patient, Gait study 1.051   PMID:17953624 1460000000.0 E 022f77d8-cf7c-5d5c-99ba-be8a60d6ca28 male 65 182.0 83.0   0.0 0.0 7.07
121 JuCo11 Control Patient, Gait study 1.203   PMID:17953624 1460000000.0 E 4a853aed-c79e-53e3-b681-46347713332e male 72 171.0 70.0   0.0 0.0 10.6
122 JuCo12 Control Patient, Gait study 1.143   PMID:17953624 1460000000.0 E 583b6504-ffd7-5587-a51e-4394bd2a181d female 72 160.0 61.0   0.0 0.0 8.94
123 JuCo13 Control Patient, Gait study 1.17   PMID:17953624 1460000000.0 E 3b16ab97-68a5-5893-a18b-9cc0fdd0b2a1 male 61 177.0 72.0   0.0 0.0 9.97
124 JuCo14 Control Patient, Gait study 1.42   PMID:17953624 1460000000.0 E 186c6cc4-8c39-536d-bd0b-76a72201ec54 male 55 180.0 90.0   0.0 0.0  
125 JuCo15 Control Patient, Gait study 1.164   PMID:17953624 1460000000.0 E e109d040-cfe1-5f1a-9c94-e4139a69f8c2 female 53 150.0 56.0   0.0 0.0  
126 JuCo16 Control Patient, Gait study 1.484   PMID:17953624 1460000000.0 E f6818777-2c4a-5b9e-8f7c-627751fd521b male 66 172.0 83.0   0.0 0.0 7.13
127 JuCo17 Control Patient, Gait study 1.54   PMID:17953624 1460000000.0 E d3074dec-7f39-583e-af54-1c1481e66c0c female 65 170.0 60.0   0.0 0.0 7.06
128 JuCo18 Control Patient, Gait study 1.231   PMID:17953624 1460000000.0 E 325bd026-f08b-5c5b-bca3-7f32d4be7064 female 64 163.0 59.0   0.0 0.0 9.85
129 JuCo19 Control Patient, Gait study 1.253   PMID:17953624 1460000000.0 E 2dc9bc33-c1a6-5dfe-ae04-8b2f678fbee0 female 59 168.0 59.0   0.0 0.0 8.91
130 JuCo20 Control Patient, Gait study 1.031   PMID:17953624 1460000000.0 E 6538b222-4103-5e98-b24a-0c6c1e1f3167 female 67 160.0 67.0   0.0 0.0 13.4
131 JuCo21 Control Patient, Gait study 1.13   PMID:17953624 1460000000.0 E b12d8e07-18d8-5e8c-9a54-194823923266 male 60 185.0 88.0   0.0 0.0 10.45
132 JuCo22 Control Patient, Gait study 1.144   PMID:17953624 1460000000.0 E 19d65d5e-cf60-5033-bbd2-55887990da8f female 62 162.0 75.0   0.0 0.0 11.98
133 JuCo23 Control Patient, Gait study 1.17   PMID:17953624 1460000000.0 E d1a7d028-1947-5f28-a665-5a7ddf75452c male 68 182.0 87.0   0.0 0.0 12.02
134 JuCo24 Control Patient, Gait study 1.391   PMID:17953624 1460000000.0 E b9f988b3-5832-5dbe-8dfb-eaf7e6c07176 male 64 170.0 72.0   0.0 0.0 8.84
135 JuCo25 Control Patient, Gait study 1.265   PMID:17953624 1460000000.0 E 0cd12385-aa26-56ba-9656-b6c95500e209 male 53 172.0 85.0   0.0 0.0 8.44
136 JuCo26 Control Patient, Gait study 1.349   PMID:17953624 1460000000.0 E deb3d959-e9a8-5379-939b-8962608adf33 male 60 167.0 70.0   0.0 0.0 8.28
137 SiCo01 Control Patient, Gait study 0.906   PMID:15929090 1460000000.0 E a8f2d2f6-8d27-5867-a049-a0b2b6c2c983 female 53 1.53 56.0       12.24
138 SiCo03 Control Patient, Gait study 1.371   PMID:15929090 1460000000.0 E 2d9d2953-a9c2-5ca1-993c-9cb48b0137f4 female 65 1.72 60.0       7.49
139 SiCo04 Control Patient, Gait study 1.427   PMID:15929090 1460000000.0 E b0105184-2f66-50f2-8b97-d58b75696e0c male 71 1.77 80.0       7.16
140 SiCo05 Control Patient, Gait study 1.07   PMID:15929090 1460000000.0 E d223adca-63d2-510c-9c79-3455195f04e1 female 55 1.56 90.0       10.01
141 SiCo06 Control Patient, Gait study 1.263   PMID:15929090 1460000000.0 E acc6e311-d88f-5dd7-b6cd-9df1ab29cce9 female 70 1.56 54.0       9.69
142 SiCo07 Control Patient, Gait study 1.348   PMID:15929090 1460000000.0 E 5aa58743-8a62-5d4e-b9e4-969ecd756085 female 57 1.59 55.0       9.99
143 SiCo08 Control Patient, Gait study 1.191   PMID:15929090 1460000000.0 E 4a81f067-d66f-55b4-87a6-e814db878dd3 female 65 1.57 69.0       12.19
144 SiCo09 Control Patient, Gait study 1.31   PMID:15929090 1460000000.0 E 5c61449d-255f-52cf-b2f7-f0a45d3bd35f male 56 1.72 75.0       9.47
145 SiCo10 Control Patient, Gait study 1.442   PMID:15929090 1460000000.0 E fd69e6d4-5beb-5f21-aae0-c36cf7ca1221 male 50 1.7 68.0       8.79
146 SiCo11 Control Patient, Gait study 1.465   PMID:15929090 1460000000.0 E 2cf0ced0-4df5-51d5-a127-cf1053617cbb female 60 1.6 80.0       8.97
147 SiCo12 Control Patient, Gait study 1.538   PMID:15929090 1460000000.0 E 44ca2391-7cd4-52b9-a7ef-181313e2d183 female 37 1.61 54.0       6.23
148 SiCo13 Control Patient, Gait study 1.299   PMID:15929090 1460000000.0 E 402ce7ef-7911-5568-bc98-1308e6a22ab1 male 61 1.73 69.0       9.97
149 SiCo14 Control Patient, Gait study 1.151   PMID:15929090 1460000000.0 E 18e08413-5dda-5b4e-8c90-2c3f01676cb7 male 52 1.64 64.0       9.77
150 SiCo15 Control Patient, Gait study 1.469   PMID:15929090 1460000000.0 E 3c4c896c-e738-591d-8402-c9121f2556fa female 54 1.69 73.0       7.68
151 SiCo16 Control Patient, Gait study 1.073   PMID:15929090 1460000000.0 E 65c50ca7-80a2-55fc-b223-c51bcd6dd734 male 60 1.85 88.0       9.28
152 SiCo17 Control Patient, Gait study 1.202   PMID:15929090 1460000000.0 E a20d4a51-860b-5dfe-92fc-4a1e8f3c5537 female 64 1.65         10.44
153 SiCo18 Control Patient, Gait study 1.27   PMID:15929090 1460000000.0 E 06546342-ba79-5358-aafc-d99c698b11e4 female 61 1.56 56.0       10.41
154 SiCo19 Control Patient, Gait study 1.086   PMID:15929090 1460000000.0 E d5b4729c-68b9-59e6-adef-30ad1c2df3a0 male 51 1.67 80.0       10.04
155 SiCo20 Control Patient, Gait study 1.25   PMID:15929090 1460000000.0 E 060072ee-a7e2-598c-82b2-adcda8f2fbd3 male 62 1.72 86.0       9.7
156 SiCo21 Control Patient, Gait study 1.458   PMID:15929090 1460000000.0 E 2f4ef62a-17c7-5977-b66a-46ecd40dad25 male 56 1.78 98.0       7.63
157 SiCo22 Control Patient, Gait study 1.418   PMID:15929090 1460000000.0 E 0d820a4c-bf87-57c0-98d6-bf26f8f22f49 male 52 1.66 88.0       7.63
158 SiCo23 Control Patient, Gait study 1.16   PMID:15929090 1460000000.0 E c5ae6592-54d4-5024-83e2-16e2340a8495 male 53 1.7 76.0       8.94
159 SiCo24 Control Patient, Gait study 1.03   PMID:15929090 1460000000.0 E 413800ee-59fa-5751-81bc-b45b158f47ca male 54 1.7 73.0       11.05
160 SiCo25 Control Patient, Gait study 1.13   PMID:15929090 1460000000.0 E 2b5ffc7e-a0a0-5814-a572-82dbd42ef386 male 57 1.76 80.0       9.16
161 SiCo26 Control Patient, Gait study 1.0   PMID:15929090 1460000000.0 E 0c9de025-71ac-5584-9e07-3964543df805 male 60 1.74 72.0       9.2
162 SiCo27 Control Patient, Gait study 1.12   PMID:15929090 1460000000.0 E e48ba217-3cfa-5ef1-b192-508a3042b23b male 67 1.9 95.0       12.52
163 SiCo28 Control Patient, Gait study 0.99   PMID:15929090 1460000000.0 E e0ccefa4-41c1-5f49-809e-2a47b43a68d4 male 61 1.7 72.0       12.65
164 SiCo29 Control Patient, Gait study 1.29   PMID:15929090 1460000000.0 E 54052b3b-1cab-5033-90af-fda6deec348d male 53 1.7 87.0       11.41
165 SiCo30 Control Patient, Gait study 1.42   PMID:15929090 1460000000.0 E 7106b614-743c-543c-bc42-fcbee0502d17 male 63 1.74 82.0       8.68

We see that the data was read into a dataframe, and that we read all the 166 rows and 16 columns of data. However, it is also possible to read the data partially, which is useful in the prescence of large datasets.

1
2
3
4
5
6
7
 # get only the first 5 rows
 data = tb.get_data(limit=5)

 # print rows and column information for the data
 print("Data has {} rows and {} columns").format(len(data.index),len(data.columns))

 data
Data has 5 rows and 16 columns
data
  id subject_type speed_01 speed_10 reference v_lastmodified_epoch v_status v_uuid gender age height weight hoehnyahr updrs updrsm tuag
0 GaPt03 Parkinsons Patient, Gait study   0.778 PMID:16176368 1460000000.0 E 69cabe6a-8b75-5fa1-ab43-5a15ab118b26 female 82 1.45 50.0 3.0 20.0 10.0 36.34
1 GaPt04 Parkinsons Patient, Gait study 0.642 0.818 PMID:16176368 1460000000.0 E c57a9ac1-6388-5b38-9f24-94ffa725be36 male 68 1.71   2.5 25.0 8.0 11.0
2 GaPt05 Parkinsons Patient, Gait study 0.908 0.614 PMID:16176368 1460000000.0 E 2b0b804d-5a74-5fc8-a641-f9f2a9d5b578 female 82 1.53 51.0 2.5 24.0 5.0 14.5
3 GaPt06 Parkinsons Patient, Gait study 0.848 0.937 PMID:16176368 1460000000.0 E d665e2a2-3a77-57c6-a091-dafda73929c6 male 72 1.7 82.0 2.0 16.0 13.0 10.47
4 GaPt07 Parkinsons Patient, Gait study 0.677 0.579 PMID:16176368 1460000000.0 E 78a1315f-1a0e-55fa-8be6-279b561fddaa female 53 1.67 54.0 3.0 44.0 22.0 18.34

We see that in this case we only got the first 5 rows of data, because we specified that number of rows using the limit parameter for get_data().

What if we wanted to only take rows 20 through 30? This would be possible by specifying a limit of 10, to get 10 rows, and an offset of 20, to start getting the data in the 20th row. This is demonstrated as follows.

1
2
3
4
5
6
7
 # get only the first 5 rows
 data = tb.get_data(limit=10, offset=20)

 # print rows and column information for the data
 print("Data has {} rows and {} columns").format(len(data.index),len(data.columns))

 data
Data has 10 rows and 16 columns
data
  id subject_type speed_01 speed_10 reference v_lastmodified_epoch v_status v_uuid gender age height weight hoehnyahr updrs updrsm tuag
0 GaPt24 Parkinsons Patient, Gait study 1.255   PMID:16176368 1460000000.0 E a101da76-087f-508f-828a-534e8ea2ac39 male 68 1.72 73.0 2.5 42.0 15.0 11.42
1 GaPt25 Parkinsons Patient, Gait study 1.128   PMID:16176368 1460000000.0 E 3262c86a-8fb1-58e3-a775-390037e01710 male 81 1.76 90.0 2.5 31.0 18.0 15.22
2 GaPt26 Parkinsons Patient, Gait study 1.244   PMID:16176368 1460000000.0 E bba41392-f156-5711-8448-34bdc77d3de8 female 78 1.52 60.0 2.5 24.0 5.0 7.27
3 GaPt27 Parkinsons Patient, Gait study 1.423   PMID:16176368 1460000000.0 E 47e0d47c-a884-5a6a-af80-1c8786131e3e male 72 1.8 95.0 2.0 21.0 10.0 7.88
4 GaPt28 Parkinsons Patient, Gait study 0.987   PMID:16176368 1460000000.0 E 468a9868-6f93-5329-ac50-0d3a95254a11 male 61 1.79 101.0 2.5 54.0 29.0 13.02
5 GaPt29 Parkinsons Patient, Gait study 1.092   PMID:16176368 1460000000.0 E 0c7e8115-e496-5ab1-b723-2bfbedd0e2b7 male 68 1.63 80.0 2.0 27.0 16.0 10.16
6 GaPt30 Parkinsons Patient, Gait study 1.064   PMID:16176368 1460000000.0 E 27656eaf-cfa0-5b31-8105-124d60e193f5 male 69 1.78 93.0 2.0 20.0 12.0 9.91
7 GaPt31 Parkinsons Patient, Gait study 0.876   PMID:16176368 1460000000.0 E 0b5bf70a-6720-58c2-965b-cdc061f0d991 male 67 1.76 90.0 2.5 27.0 13.0 12.6
8 GaPt32 Parkinsons Patient, Gait study 1.242   PMID:16176368 1460000000.0 E e7c80672-61fa-599b-8748-8ee97a511edd male 63 1.69 75.0 2.0 33.0 24.0 11.22
9 GaPt33 Parkinsons Patient, Gait study 0.825   PMID:16176368 1460000000.0 E c8c96583-df0b-5050-b2a3-efcc20f8d49e male 63 1.86 80.0 2.5 42.0 31.0 11.97

Note

The maximum default value for limit is 1000. In order to get larger chunks of data we recomend using the get_data_iter() method, which gets the gata in an iterative manner. The method is fully described in the tabular section of the Data Models page.

Analyzing the Data

As we have seen, the python client allows to get the data in a format that is flexible and easy to use. We now show a very simple example for plotting the data that we already have.

Note

In order to plot the data as shown in the following part of the tutorial, you need to have installed matplotlib.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
 import matplotlib.pyplot as plt
 import pandas as pd
 import numpy as np

 # get all the data
 data = tb.get_data()

 # give the index a name (ind)
 data['ind']= data.index

 # define x and y variables (and get rid of undefined entries)
 x=data['updrs'].fillna(0)
 y=data['updrsm'].fillna(0)
 plt.scatter(x,y, color='c')

 z = np.polyfit(x, y, 1)
 p = np.poly1d(z)

 plt.plot(x,p(x),"r--")

 # adjust axes of plot and add labels
 axes = plt.gca()
 axes.set_title('UPDRSM vs. UPDRS')
 axes.set_xlabel('UPDRS'); axes.set_ylabel('UPDRSM')

 plt.show()
../_images/tabular_15_0.png

Note

The reason for presenting the example above is to illustrate how simple it can be to work with the downloaded dataframe. This is just a very easy example to get you stated in the analysis and exploration of your tabular datasets.