Access data table name in Workflow/ loop over data tables

ONE DATA Core
1

Dear OD users,
I would like to process several similar datasets with the same workflow to calculate basic statistics. The result should be saved in one table for further processing. Therefore it would be nice to

  1. loop over the list of datasets and/or
  2. access a unique label of the dataset (eg. the data table name) as a tag in a new column

What would be the best approach to do something like this?
Thanks,
Frauke

2

You could create a workflow that uses a ONE DATA type key to retrieve the list of data tables via the server’s own API (connect to http://localhost:8080/api/v1/data and execute a GET request according to http://od-api.pages.intranet.onelogic.de/#data_get). Use Flexible REST API and XML / JSON Parsing v1.2 processors. You can then use again Flexible REST API to invoke another workflow via ONE DATA’s API that calculates your statistics per data table that is of interest.
→ http://od-api.pages.intranet.onelogic.de/#workflows__workflowid__jobs_post

The respective API calls can also be intercepted while using OD. Relevant pages are the data table overview page of a project (https://internal.onedata.de/onedata/projects/0b72d588-0a56-45a0-ac4c-af2d699ab20e/details?tab=Data%20Tables) to see how data tables can be listed and the API calls that are triggered by Save & Run in a workflow.

1 Like
3

thanks, stefan! i will give this a try.