-
Notifications
You must be signed in to change notification settings - Fork 20
Closed
Description
cargo install datafusion-dft --git https://github.com/datafusion-contrib/datafusion-dft --rev b1f1011d9b1070eec658d51992ab717709e355f0 --features "deltalake s3 functions-json functions-parquet"
Updating git repository `https://github.com/datafusion-contrib/datafusion-dft`
Ignored package `datafusion-dft v0.2.2 (https://github.com/datafusion-contrib/datafusion-dft?rev=b1f1011d9b1070eec658d51992ab717709e355f0#b1f1011d)` is already installed, use --force to overrideI have set up S3 as an object store.
2025-06-03T12:02:55Z DEBUG datafusion_dft::config] Config exists
[2025-06-03T12:02:55Z DEBUG datafusion_dft::config] Parsed config: AppConfig { shared: ExecutionConfig { object_store: Some(ObjectStoreConfig { s3: Some([S3Config { bucket_name: "bucket", object_store_url: Some("s3://bucket"), aws_access_key_id: Some("redact"), aws_secret_access_key: Some("redact"), _aws_default_region: None, aws_endpoint: Some("https://s3.us-east-1.amazonaws.com"), aws_session_token: None, aws_allow_http: None }]) }), ddl_path: Some("/home/jovyan/.config/dft/ddl.sql"), benchmark_iterations: 10, datafusion: None, dedicated_executor_enabled: false, dedicated_executor_threads: 8, catalog: CatalogConfig { name: "dft" } }, cli: CliConfig { execution: ExecutionConfig { object_store: Some(ObjectStoreConfig { s3: Some([S3Config { bucket_name: "bucket", object_store_url: Some("s3://bucket"), aws_access_key_id: Some("redact"), aws_secret_access_key: Some("redact"), _aws_default_region: None, aws_endpoint: Some("https://s3.us-east-1.amazonaws.com"), aws_session_token: None, aws_allow_http: None }]) }), ddl_path: Some("/home/jovyan/.config/dft/ddl.sql"), benchmark_iterations: 10, datafusion: None, dedicated_executor_enabled: false, dedicated_executor_threads: 8, catalog: CatalogConfig { name: "dft" } } }, tui: TuiConfig { execution: ExecutionConfig { object_store: None, ddl_path: Some("/home/jovyan/.config/dft/ddl.sql"), benchmark_iterations: 10, datafusion: None, dedicated_executor_enabled: false, dedicated_executor_threads: 8, catalog: CatalogConfig { name: "dft" } }, display: DisplayConfig { frame_rate: 30.0 }, interaction: InteractionConfig { mouse: false, paste: false }, editor: EditorConfig { experimental_syntax_highlighting: false } }, db: DbConfig { path: Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/home/jovyan/.local/share/dft/", query: None, fragment: None } } }
[2025-06-03T12:02:55Z INFO datafusion_app::extensions::s3] S3 configs exists
[2025-06-03T12:02:55Z INFO object_store::aws::builder] Using Static credential provider
[2025-06-03T12:02:55Z DEBUG datafusion_app::extensions::s3] created object store: AmazonS3(bucket)
[2025-06-03T12:02:55Z INFO datafusion_app::extensions::s3] registered s3 object store at s3://bucketBut by DDL query is failing
[2025-06-03T12:02:55Z INFO datafusion_app::local] Loading DDL from: Some("/home/jovyan/.config/dft/ddl.sql")
[2025-06-03T12:02:55Z DEBUG datafusion_app::local] Executing DDL statement: "CREATE EXTERNAL TABLE tbl \nSTORED AS DELTATABLE \nLOCATION 's3://bucket/delta_traces_3/otel_traces/'"
[2025-06-03T12:02:55Z ERROR datafusion_app::local] Error executing DDL statement: External error: Cannot infer storage location from: Unknown scheme: s3. Known schemes: file,memoryMetadata
Metadata
Assignees
Labels
No labels