top of page

Data Automation and Integration Engine

Extract - Transform - Load. Thousands of APIs are waiting to serve data.

​

Enjoy powerful data processing without cost constraints:

  • no charges based on data volume,

  • no limits on data update intervals,

Install InDriver, and leverage machine performance limits.

ETL/ELT

ETL, or Extract, Transform, Load, is a pivotal process in the world of data management. It involves extracting raw data from various sources, transforming it into a consistent format, and loading it into a destination for analysis or storage. ETL plays a vital role in business intelligence, enabling organizations to consolidate and harmonize diverse datasets, ultimately empowering informed decision-making and strategic insights.

INPUT DATA
[TB] Terabytes of raw and unstructured data

Basic ETL Solution in InDriver JS Engine Task

OUTPUT DATA
Valuable business data: Time synchronized, Interpolled, Aggreated, Calculated, Filtered, Sorted, Processed... Transformed

Example: Basic ETL Solution in InDriver JS Engine Task

InDriver simplifies access to RESTful API Resources with just one line of JS code. Define requests, set intervals, and effortlessly process returned JSON data—ideal for logging into any database. No limits on requests or call frequency, making InDriver a reliable, real-time data pump for applications, dashboards, and analytics. A robust alternative to SaS ETL Tools like Airbyte, Fivetran, MuleSoft Anypoint Platform, Supermetrics, and Workato. Free for non-commercial use, commercially licensed per machine with unlimited data transfer and connectors. The powerful JS Engine enables easy implementation of RESTAPI requests, synchronized source combinations, data processing, analytics, and SQL database logging. Download InDriver and power up your Data Integration Engine.

Business Use Cases of Data Automation

Data automation can streamline various business processes and improve efficiency. Here's a list of data automation business cases across different industries:

  1. Data Entry and Processing Automation:

    • Automate manual data entry tasks to reduce errors and save time.

    • Automatically process forms, invoices, and other documents.

  2. Customer Relationship Management (CRM):

    • Automate data updates and syncing between CRM and other systems.

    • Trigger automated responses based on customer interactions.

  3. Sales and Marketing Automation:

    • Implement automated lead scoring and qualification.

    • Automate email marketing campaigns based on customer behavior.

  4. Financial Data Automation:

    • Automate data entry and reconciliation in accounting systems.

    • Implement automated financial reporting and analysis.

  5. Supply Chain Management:

    • Automate inventory tracking and reorder processes.

    • Implement demand forecasting using automated data analysis.

  6. Human Resources:

    • Automate the onboarding and offboarding processes.

    • Implement automated payroll and benefits processing.

  7. Business Intelligence and Analytics:

    • Automate data collection and cleansing for business intelligence.

    • Implement automated reporting and dashboard updates.

  8. Quality Assurance and Testing:

    • Automate data validation and quality checks.

    • Implement automated testing in software development.

  9. Compliance and Regulatory Reporting:

    • Automate the generation of compliance reports.

    • Ensure data accuracy and timeliness in regulatory submissions.

  10. Healthcare Data Automation:

    • Implement automated patient record updates.

    • Streamline medical billing processes through automation.

  11. E-commerce and Retail:

    • Automate inventory management and order processing.

    • Implement personalized marketing based on customer behavior.

  12. Manufacturing Process Automation:

    • Automate data collection from sensors and production equipment.

    • Implement predictive maintenance through automated data analysis.

  13. IT Operations and Monitoring:

    • Automate system monitoring and alerting.

    • Implement automated incident response and resolution.

  14. Social Media and Sentiment Analysis:

    • Automate social media data collection and analysis.

    • Implement sentiment analysis for brand monitoring.

  15. Energy Management:

    • Automate energy consumption monitoring and reporting.

    • Implement automated alerts for energy efficiency improvements.

  16. Legal Document Automation:

    • Automate document creation and review processes.

    • Implement contract management through automation.

  17. Education and Learning Management:

    • Automate grading and assessment processes.

    • Implement personalized learning paths through automation.

  18. Customer Support:

    • Implement chatbots for automated customer support.

    • Automate ticket routing and resolution processes.

  19. Real Estate:

    • Automate property management tasks such as rent collection.

    • Implement automated reporting for property portfolios.

  20. Cybersecurity:

    • Automate threat detection and response.

    • Implement automated patch management for security updates.

​

These are just examples, and the specific business cases for data automation will depend on the unique needs and goals of each organization. Implementing data automation can lead to increased efficiency, reduced errors, and improved decision-making across various business functions.

InDriver JavaScript Task Code Examples

Open Weather Map
https://openweathermap.org

openweather logo

Connect to https://openweathermap.org/ and retrieve the weather update for a city of Krakow (Poland).

  • onStartup

 

InDriver.import("RestApi");

 

RestApi.defineRequest('krakow','{"url":"https://api.openweathermap.org/data/2.5/weather?appid=your_app_id&q=krakow","timeout":5000, "type":"get","headers":{"ContentTypeHeader":"application/json"}}');

InDriver.installHook(60000);

 

  • onHook
     

RestApi.sendRequest('krakow');
if (RestApi.isSucceeded()) {

        const data = RestApi.getData('krakow');

        ...

        let ts = InDriver.hookTs();

InDriver.sqlExecute("azureserver", "insert into public.weather (source, timestamp, data )
values ( 'krakow','"+ts.toISOString()+"',$$"+data+"$$);"

}

  • Output

 

data = {"coord":{"lon":19.9167,"lat":50.0833},"weather":[{"id":701,"main":"Mist","description":"mist","icon":"50n"}],"base":"stations","main":{"temp":272.54,"feels_like":270.64,"temp_min":271.94,"temp_max":273.7,"pressure":1018,"humidity":88},"visibility":1100,"wind":{"speed":1.54,"deg":200},"clouds":{"all":100},"dt":1701968996,"sys":{"type":2,"id":2074307,"country":"PL","sunrise":1701930268,"sunset":1701959943},"timezone":3600,"id":3094802,"name":"Krakow","cod":200}

YouTube
https://www.youtube.com

youtube logo

Connect to https://www.googleapis.com and retrieve video data updates,
eg. videos from the Innovative Data Analytics channel.

  • onStartup

 

InDriver.import("RestApi");

 

RestApi.defineRequest('youtube','{"url":"https://www.googleapis.com/youtube/v3/channels?part=snippet,contentDetails,statistics&id=UC2609bKrdZz_haQTfMwTGRg&key=Your_API_Key","timeout":5000, "type":"get","headers":{"ContentTypeHeader":"application/json"}}');

​

InDriver.installHook(60000);

 

  • onHook
     

RestApi.sendRequest('youtube');
if (RestApi.isSucceeded()) {

        const data = RestApi.getData('youtube');

        ...

        let ts = InDriver.hookTs();

InDriver.sqlExecute("azureserver", "insert into public.youtube(source, timestamp, data )
values ( 'youtube','"+ts.toISOString()+"',$$"+data+"$$);"

}

  • Output

 

data = { "kind": "youtube#channelListResponse","etag": "wHk1oA0htOMVWdDdABeEjLvMSF4","pageInfo": {"totalResults": 1,"resultsPerPage": 5},"items": [{"kind": "youtube#channel","etag": "jGPYGXDHdpd3KQK9R1C5MX4y10I","id": "UC2609bKrdZz_haQTfMwTGRg","snippet": {"title": "Innovative Data Analytics","description": "Welcome to the inanalytics.io YouTube channel – your go-to destination for mastering data automation with inDriver, your specialized JavaScript engine. Unleash the power of InDriver as we bring you insightful videos and tutorials on configuring REST API to SQL pipelines, creating dynamic data automation, and maximizing the potential of this user-friendly tool. Whether you're a seasoned developer or just starting out, our content is tailored to help you streamline data automation effortlessly. Subscribe now to stay updated, explore free plans, and embark on a journey of continuous data acquisition, processin...haQTfMwTGRg"}},"statistics": {"viewCount": "117","subscriberCount": "6","hiddenSubscriberCount": false,"videoCount": "4"}}}

Polygon.io
https://www.polygon.io

polygon logo

Connect to https://www.polygon.io and retrieve stock exchange rates, in the example below, the request gets aggregate bars for Apple over the last day.

  • onStartup

 

InDriver.import("RestApi");

 

RestApi.defineRequest('polygon','{"timeout":5000, "type":"get","headers":{"ContentTypeHeader":"application/json"}}');

​

InDriver.installHook(60000);

In this example, Request Url is created dynamically for every onHook function.

 

  • onHook
     

let ts = InDriver.hookTs(10000);

let dt = new Date(ts);

let df = new Date();

df.setDate(dt.getDate()-1);

let req = '{"url":"https://api.polygon.io/v2/aggs/ticker/AAPL/range/1/day/'+df.toISOString().split("T")[0]+'/'+dt.toISOString().split("T")[0]+'?adjusted=true&sort=asc&limit=120&apiKey=Your_API_Key"}';

​

RestApi.sendRequest('polygon',req);

​

);
if (RestApi.isSucceeded()) {

        const data = RestApi.getData('polygon');

        ...

        let ts = InDriver.hookTs();

InDriver.sqlExecute("azureserver", "insert into public.polygon(source, timestamp, data )
values ( 'AAPL','"+ts.toISOString()+"',$$"+data+"$$);"

}

  • Output

 

data = {"ticker":"AAPL","queryCount":1,"resultsCount":1,"adjusted":true,"results":[{"v":4.7413955e+07,"vw":194.3967,"o":193.63,"c":194.27,"h":195,"l":193.59,"t":1701925200000,"n":495048}],"status":"DELAYED","request_id":"5c16ef0d2d08bac52fd892538d700f30","count":1}

bottom of page