Database Connection Runtime Behavior & Best Practice

[zoomingrocket]

Hi Hop community,
We’re using Apache Hop in production for enterprise data integration and have a couple of questions about Oracle database connection behavior and resilience patterns. Would appreciate guidance and/or pointers!

1. How are database connections managed across transforms?
   We understand Hop doesn’t do connection pooling by default.
   If multiple transforms in the same pipeline use the same metadata connection (same name/config), does Hop: open a separate physical DB connection per transform/thread, or
   reuse a single connection across those transforms?

Are there differences in behavior between:
transforms within the same pipeline vs. across sub-pipelines (Pipeline Executor) or workflows?
parallel “copies” of the same transform (e.g., when Copies > 1)?

Any configuration flags or best practices to minimize connection churn without introducing contention (e.g., “one connection per transform copy” vs. “shared connection per pipeline”)?

2. Handling transient network/DB errors (e.g., I/O Connection Reset)
   We’re seeing intermittent errors like java.sql.SQLRecoverableException: I/O Error: Connection reset (or driver-specific equivalents).
   The “Database” transform-level error handling doesn’t seem to catch these low-level I/O exceptions; the pipeline terminates without giving us a chance to retry gracefully.
   What are the recommended patterns in Hop to do controlled retries/backoff for transient DB issues?

Is there a built-in way to auto-retry a transform or reconnect the DB handle?
Are there any try/catch patterns that allow us to do graceful retry and a Delay step to implement exponential backoff?
Any examples using Abort, Repeat, or Row-level error handling that do catch these exceptions?

On the driver level, we are already set up for client-side optimal timeouts and keepalive but as in the distributed cloud world, network hiccups are normal, so looking for a better approach

Thanks!

[bamaer]

1. Each transform in a pipeline uses and needs its own connection. Connection pooling wouldn't help a lot (if at all) here. You can't read from a single query or write to a single query over a pool of connections.
2. A best practice is to design for failure and make your pipelines as idempotent as possible. There's no auto-retry functionality in pipelines, but in many cases that wouldn't make a lot of difference (again, if at all).