Data passing SDK¶
Full documentation of the Orchest SDK can be found here: orchest-sdk.
Orchest SDK for data passing between pipeline steps in the Orchest platform. The SDK manages the destination and source of the data, leaving the user only with the decision what data to send (because receiving automatically retrieves all the sent data).
The destination and source of the data are inferred through the defined pipeline definition in the platform (the pipeline.json file).
We plan to also support other popular programming languages such as R.
Python package to pass data between pipeline steps in the Orchest platform.
The SDK comes pre-installed when using the Orchest platform.
Currently the recommended method for installing the Orchest SDK is through the GitHub repository
# To get the latest release you can substitute "master" for "develop". pip install git+https://github.com/orchest/orchest-sdk.git@master#subdirectory=python
Example of passing data, where the pipeline (defined inside the
Step 1 -> Step 2.
"""Step 1""" from orchest import transfer data = [1, 2, 3] # Output the data so that Step 2 can retrieve it. transfer.output(data)
"""Step 2""" from orchest import transfer # Get the input for Step 2, i.e. the output of Step 1. data = transfer.get_inputs() # data = [[1, 2, 3]]