pleasefixai/oracle-ai-data-platform-workbench-spark-connectors
Guides to connect Oracle AI Data Platform notebooks to 26 data sources with Spark.
Connect from an AIDP notebook to Oracle AI Lakehouse (ALH), Autonomous Data Warehouse (ADW), or Autonomous Transaction Processing (ATP). Prefer the AIDP aidataplatform Spark format handler for wallet/password and catalog.id paths (ORACLE_ALH / ORACLE_ATP), but use raw Spark JDBC for IAM DB-token because the AIDP connector does not support DB-token auth yet.
Read and write AWS S3 (s3a://) from an AIDP notebook. Use when the user mentions S3, AWS S3 bucket, s3a, or has AWS access keys. Auth is access key + secret key via the Hadoop S3A connector. boto3 is also available for non-Spark management operations (list, copy).
Read and write Azure Data Lake Storage Gen2 (abfss://) from an AIDP notebook. Use when the user mentions ADLS, Azure Data Lake, abfss, or wants to ingest from a multi-cloud Azure source. Auth is OAuth client-credentials (Service Principal client_id + secret + tenant).
Read or write Azure SQL Database from an AIDP notebook through the AIDP aidataplatform Spark format handler. Use when the user mentions Azure SQL, Azure SQL Database, AZURE_SQLSERVER, or a database.windows.net endpoint. Auth is SQL username/password.
Help the user pick the right connector skill for their data source from an AIDP notebook. Use as a router when the user mentions multiple sources, isn't sure which connector applies, or asks "how do I connect to X from AIDP". Covers 26 data sources — Oracle Autonomous DB family (ALH/ADW/ATP), generic Oracle DB, ExaCS, PeopleSoft, Siebel, Fusion ERP/BICC, EPM Cloud, Essbase, OCI Streaming, Object Storage, Iceberg, plus PostgreSQL, MySQL/HeatWave, Db2, SQL Server, Azure SQL, Hive, Snowflake, Azure ADLS, AWS S3, Salesforce, NetSuite, generic REST, custom JDBC, Excel.
Read or write IBM Db2 from an AIDP notebook through the AIDP aidataplatform Spark format handler. Use when the user mentions DB2, IBM Db2, LUW, or type=DB2. Auth is host/port + database name + user/password.
Run a Planning data-slice export against Oracle EPM Cloud (Planning / EPBCS) and materialize as a Spark DataFrame in an AIDP notebook. Use when the user mentions EPM Cloud, EPBCS, Hyperion Planning, planning app, MDX export, or wants Planning data in Spark. HTTP Basic auth with identity-domain-prefixed username.
Run an MDX query against an Oracle Essbase 21c cube and materialize the result as a Spark DataFrame in an AIDP notebook. Use when the user mentions Essbase, MDX, Essbase 21c, OLAP cube, or wants to read cube data into Spark. Auth is HTTP Basic.
Read or write Oracle Exadata Cloud Service (ExaCS) from an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions ExaCS, Exadata, Exadata Cloud, RAC SCAN listener, or has a private-subnet Oracle DB. Prefer the official ORACLE_EXADATA connector sample with catalog.id, pushdown.sql, and connector write modes.
Read Excel (.xlsx, .xls) files into a Spark DataFrame from an AIDP notebook. Use when the user mentions Excel, .xlsx, .xls, or has spreadsheet files in a Volume / Object Storage bucket. Two paths — the com.crealytics.spark.excel Spark format (cluster jar required) and a pandas → CSV → spark.read.csv fallback that needs no jars.
Pull a Fusion BICC bulk extract into a Spark DataFrame from an AIDP notebook. Use when the user mentions BICC, Fusion bulk extract, BI Cloud Connector, PVO, or needs >50k rows from Fusion. The recommended path uses AIDP's built-in spark.read.format("aidataplatform") connector (matches the official Oracle AIDP sample). HTTP Basic auth.
Pull data from Oracle Fusion ERP / HCM / SCM REST APIs into a Spark DataFrame from an AIDP notebook. Use when the user mentions Fusion ERP, Fusion REST API, FA REST, Cloud ERP, or wants live data from a Fusion pod. HTTP Basic auth only. For volumes >499 rows/page or bulk extracts, route to aidp-fusion-bicc.
Read or write Apache Hive from an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions Hive, HiveServer2, HS2, HCatalog, or has a Hive metastore host/port. Auth is host/port + user/password. Read-write.
Read and write Apache Iceberg tables backed by OCI Object Storage from an AIDP notebook. Use when the user mentions Iceberg, Apache Iceberg, time travel, snapshots, schema evolution, partition evolution, or wants ACID transactions on data lake files. Uses the Iceberg Hadoop catalog on oci:// — auth is implicit via the workspace IAM identity.
Connect to ANY database that has a JDBC driver from an AIDP notebook using Spark's native format("jdbc"). Use when the user mentions a DB without a dedicated AIDP connector — SQLite, ClickHouse, DuckDB, generic JDBC URL — or wants to use a custom JDBC driver they uploaded. Auth is driver-specific.
Read or write MySQL or OCI MySQL HeatWave from an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions MySQL, HeatWave, MySQL Database Service, MDS, or has a MySQL host/port. Auth is host/port + user/password.
Read NetSuite SuiteAnalytics Connect data from an AIDP notebook through the AIDP aidataplatform Spark format handler. Use when the user mentions NetSuite, SuiteAnalytics Connect, ns.account.id, ns.role.id, or ns.access.token. Supports username/password or OAuth access-token authentication. Read-only.
Read and write OCI Object Storage natively from an AIDP notebook using the oci:// URI scheme. Use when the user mentions OCI Object Storage, "oci://", external volumes, external tables backed by Object Storage, CSV/Parquet/JSON/Delta files in a bucket, or wants to land data in OCI buckets. Auth is implicit via the workspace's IAM identity — no keys in the notebook.
Read or write an Oracle Database (Compute / Base DB / on-prem / Oracle 19c, 21c, 23ai, 26ai non-Autonomous) from an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions Oracle Database, generic Oracle DB, on-prem Oracle, plain Oracle JDBC, port 1521, non-Autonomous Oracle. Read-write. Auth is host/port + database name + user/password.
Read from Oracle PeopleSoft into a Spark DataFrame in an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions PeopleSoft, PSFT, HCM, FSCM, Campus Solutions, or has a PeopleSoft host/port. Auth is host/port + database name + user/password. Read-only.
Read or write PostgreSQL from an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions PostgreSQL, Postgres, "psql", or has a Postgres host/port to connect to. Prefer the official POSTGRESQL connector sample; use native Spark JDBC only as an SSL-required fallback when the built-in connector cannot express sslmode=require.
Pull data from any REST API into a Spark DataFrame using the AIDP aidataplatform Generic REST connector. Use when the user has a non-Fusion / non-EPM / non-Essbase REST endpoint with a manifest.url describing the schema. Auth is HTTP Basic with derived properties driving query parameters.
Read from Salesforce into a Spark DataFrame in an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions Salesforce, SFDC, Sales Cloud, Service Cloud, Account, Opportunity, Lead, sObject, SOQL. Auth is host/port + user/password. Read-only.
Read from Oracle Siebel CRM into a Spark DataFrame in an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions Siebel, Siebel CRM, S_CONTACT, S_ORG_EXT, or has a Siebel host/port. Auth is host/port + database name + user/password. Read-only.
Read Snowflake from an AIDP notebook through the AIDP aidataplatform Spark format handler. Use when the user mentions Snowflake, Snowflake warehouse, Basic authentication, KeyPair authentication, private.key.file, or private.key.content. Snowflake writes are not supported in AIDP 4.0.
Read or write Microsoft SQL Server from an AIDP notebook via the AIDP aidataplatform Spark format handler. Use when the user mentions SQL Server, MSSQL, or has a TDS host/port. Auth is host/port + database + user/password.
Consume an OCI Streaming stream from an AIDP notebook via Spark structured streaming (Kafka-compat). Use when the user mentions OCI Streaming, Kafka on OCI, stream pool, structured streaming, or wants to read Kafka messages into Spark. Auth is SASL/PLAIN with an OCI auth token. Pattern matches the official Oracle AIDP sample.