Building PDA Functions
Functions are integral to the PDA. Any production grade Functions that's published and made available for public use would entail a configuration update to the entire ecosystem of PDAs. Hence, the process of developing a Function is as follows
- 1.Develop your function locally
- 2.Find an application that would use the data created by your tool or make the case of why the data created by the function would be useful to all PDA owners
- 3.Submit to Dataswift for Review
- 4.If your function is accepted, Dataswift will deploy it across PDAs.
Functions can be created independently from the PDA itself. Data scientists can develop their algorithms and experiment with data on the PDA without having to submit it for review and publishing.
You will first need to obtain a PDA for testing from the Developer's Portal. Populate your PDA with data. You can either inject your own data via API, or use one of the Data Plugs.You will also need an AWS account with your programmatic access keys enabled. This will allow you to deploy your Tool to your AWS Lambda for testing. There is a set of tools from Serverless that will help with this process.
Coding your Function
Your Tool function needs to do 3 things:
- 3.Accept data processing request which includes the current known configuration from the PDA and the bundle of data itself generating using the bundle received from (2)
A common recommendation is to split your algorithm details from the Lambda function handling details — it makes testing and debugging a lot simpler. You should be able to feed the generated Data Bundle definition into your PDA, as well as the data extracted from the PDA using the bundle into your algorithm for processing. The templates provided at the end of this document follows this recommendation.
Testing your Function
- 1.Loading and verifying the function configuration from your self-deployed Tool
- 2.Call your Tool to generate the function bundle
- 3.Authenticate with your PDA to get an access token
- 4.With the access token, register the function bundle from (2) with the PDA
- 5.Retrieve the requested data
- 6.Post the requested data into the Tool
Everytime you update your algorithm, you will need to redeploy your Tool. After which you can just do step 6 above.
While you can develop Tools in any of these popular supported languages.
- 2.Java / Scala
There are 2 templates you can make use of to scaffold your Tool. They are respectively