Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/en/developer/00-drivers/01-python.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,4 +124,4 @@ with engine.connect() as conn:
## Resources

- **PyPI**: [databend-driver](https://pypi.org/project/databend-driver/) • [databend-sqlalchemy](https://pypi.org/project/databend-sqlalchemy/)
- **GitHub**: [databend-driver](https://github.com/databendlabs/databend-py) • [databend-sqlalchemy](https://github.com/databendlabs/databend-sqlalchemy)
- **GitHub**: [databend-driver](https://github.com/databendlabs/bendsql/tree/main/bindings/python) • [databend-sqlalchemy](https://github.com/databendlabs/databend-sqlalchemy)
2 changes: 1 addition & 1 deletion docs/en/developer/00-drivers/02-nodejs.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,5 +81,5 @@ conn.close();
## Resources

- **NPM Package**: [databend-driver](https://www.npmjs.com/package/databend-driver)
- **GitHub Repository**: [databend-js](https://github.com/databendlabs/databend-js)
- **GitHub Repository**: [databend-driver](https://github.com/databendlabs/bendsql/tree/main/bindings/nodejs)
- **TypeScript Definitions**: Included in package
2 changes: 1 addition & 1 deletion docs/en/developer/00-drivers/04-rust.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Add the driver to your `Cargo.toml`:

```toml
[dependencies]
databend-driver = "0.7"
databend-driver = "0.30"
tokio = { version = "1", features = ["full"] }
```

Expand Down
3 changes: 1 addition & 2 deletions docs/en/developer/10-apis/http.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ This handler return results in `pages` with long-polling.
1. A `GET` to the `next_uri` returns the next `page` of query results. It returns `QueryResponse` too, processing it
the same way until `next_uri` is null.
2. (optional) A `GET` to the `kill_uri` to kill the query. Return empty body.
3. (optional) A `GET` to the `stats_uri` to get stats only at once (without long-polling), return `QueryResponse`
with empty `data` field.


Please note that you should keep using the latest `next_uri` to get the next page of results before the query is finished, otherwise you may miss some results or leak session resources until session timeout. The `next_uri` will be null when you have received all the results of the query.

Expand Down
51 changes: 47 additions & 4 deletions docs/en/guides/40-load-data/01-load/02-local.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,16 @@ Uploading your local data files to a stage or bucket before loading them into Da

Please note that the files must be in a format supported by Databend, otherwise the data cannot be imported. For more information on the file formats supported by Databend, see [Input & Output File Formats](/sql/sql-reference/file-format-options).

You can also load local files into tables programmatically using JDBC or Python drivers.

## Load Methods

There are two methods to load data from local files:

1. **Stage**: Upload the local file to an internal stage, then copy data from the staged file into the table. File upload occurs either through databend-query or using a presigned URL, depending on the `presigned_url_disabled` connection option (default: `false`).
2. **Streaming**: Load the file directly into the table during upload. Use this method when the file is too large to store as a single object in your object storage.


## Tutorial 1 - Load from a Local File

This tutorial uses a CSV file as an example to demonstrate how to import data into Databend using [BendSQL](../../30-sql-clients/00-bendsql/index.md) from a local source.
Expand Down Expand Up @@ -47,14 +57,48 @@ CREATE TABLE books (
Send loading data request with the following command:

```shell
❯ bendsql --query='INSERT INTO book_db.books VALUES;' --format=csv [email protected]
❯ bendsql --query='INSERT INTO book_db.books from @_databend_load file_format=(type=csv)' [email protected]
```

- The `@_databend_load` is a placeholder representing local file data.
- The [file_format clause](/sql/sql-reference/file-format-options/) uses the same syntax as the COPY command.

Alternatively, use a Python script:

```python
import databend_driver
dsn = "databend://root:@localhost:8000/?sslmode=disable",
client = databend_driver.BlockingDatabendClient(dsn)
conn = client.get_conn()
query = "INSERT INTO book_db.books from @_databend_load file_format=(type=csv)"
progress = conn.load_file(query, "book.csv")
conn.close()
```

Or use Java code:

```java
import java.sql.Connection;
import java.sql.Statement;
import java.io.FileInputStream;
import java.nio.file.Files;
import com.databend.jdbc.DatabendConnection;
String url = "jdbc:databend://localhost:8000";
try (FileInputStream fileInputStream = new FileInputStream(new File("book.csv")));
Connection connection = DriverManager.getConnection(url, "databend", "databend");
Statement statement = connection.createStatement()) {
DatabendConnection databendConnection = connection.unwrap(DatabendConnection.class);
String sql = "insert into book_db.books from @_databend_load file_format=(type=csv)";
int nUpdate = databendConnection.loadStreamToTable(sql, fileInputStream, f.length(), DatabendConnection.LoadMethod.Stage);
}
```

:::note
Be sure that you are able to connect to the backend object storage for Databend from local BendSQL directly.
If not, then you need to specify the `--set presigned_url_disabled=1` option to disable the presigned url feature.
If not, you need to specify the `--set presigned_url_disabled=1` option to disable the presigned url feature.
:::


### Step 3. Verify Loaded Data

```shell
Expand Down Expand Up @@ -96,15 +140,14 @@ CREATE TABLE bookcomments (
comments VARCHAR,
date VARCHAR
)

```

### Step 2. Load Data into Table

Send loading data request with the following command:

```shell
❯ bendsql --query='INSERT INTO book_db.bookcomments(title,author,date) VALUES;' --format=csv [email protected]
❯ bendsql --query='INSERT INTO book_db.bookcomments(title,author,date) file_format=(type=csv)' [email protected]
```

Notice that the `query` part above specifies the columns (title, author, and date) to match the loaded data.
Expand Down
Loading