Dataflow

What is dataflow?

Dataflow, or data flow, in the platform is a key component that allows users to process and transform data within an application. This is a powerful tool for integrating data, processing events and automating business processes.

Dataflow is built on the platform using a 'visual editor' of the data flow:

The visual data flow editor is an intuitive designer for creating and managing data flows. This tool allows users to sequentially create a set of Stages and Steps to implement complex data processing scripts.

• Add Stage: This is done by pressing the "+" icon on the data flow control panel. You can add an unlimited number of stages depending on the needs of your script.
• Delete Stage: To delete a stage, use the "X" button on the selected stage block.
• Edit Stage: You can only edit the name of the selected stage.
• Add Step: A new step is added by pressing the "ADD STEP" button at the appropriate stage. Users can choose step types from the proposed activity groups.
• Delete Step: To delete a step, click on the "X" icon on the selected step block.

Input Group

Components for retrieving and importing data:

• Input Steps - Overview of input steps
• Get Values from Connector - Retrieve data from connectors
• Subscribe to Connector - Subscribe to data updates
• Get Action Model - Retrieve action model
• Get Workflow Model - Retrieve workflow model
• Get Empty Model - Create empty data model
• Proxy Get Entry - Proxy entry retrieval
• Proxy Query Entry - Proxy query execution
• Get Raw Model - Retrieve raw data model
• Import File - Import data from files

Model Transformation Group

Components for data transformation and processing:

• Model Transformation Steps - Overview of transformation steps
• Transform Model - Transform data models
• Join Models - Join multiple data models
• Extract Collections - Extract collection data
• Filter Source - Filter data sources
• Lookup Reference - Reference lookup
• Execute Script - Run custom scripts
• Query Entity by Filter - Filter-based queries
• Select Many - Multiple selection
• Get Entity by ID - Retrieve by identifier
• Load Catalogs by Key - Load catalog data
• Distinct - Get distinct values
• Group By - Group data
• Calculate Array - Array calculations
• Simple Math - Mathematical operations
• Execute Dataflow - Execute nested dataflow
• Get File Info - Retrieve file information

User Contexts Group

Components for user management and authentication:

• User Contexts Steps - Overview of user context steps
• Register Context for Model - Register model context
• Register External User - External user registration
• Prepare External Keys - Prepare authentication keys
• Assign Context Role for Model - Assign roles
• Get One-Time Code for User - Generate OTP
• Confirm One-Time Code for User - Verify OTP
• Update or Create User Info - User information management
• Get User Info - Retrieve user data
• Login with Password - Password authentication
• Reset User Password Request - Password reset
• Confirm User Email Request - Email verification
• Send Templated Notification - Template-based notifications
• Send Notification - Send notifications
• Remove Assigned Roles for User - Remove user roles

Output Group

Components for data output and actions:

• Output Steps - Overview of output steps
• Store Entry over Bus - Store data via message bus
• Update Entry - Update data entries
• Deferred Update Entry - Deferred updates
• Apply Deferred Update Operations - Apply deferred operations
• Execute API Call - External API calls
• Write Response - Write response data
• Form Action - Form submission actions
• Execute Workflow - Execute workflow
• Export to File - Export data to files
• Render Template - Template rendering