Build Questions and Comments

[paultiq]

Questions and comments after spending far more time on CI than expected.

I can open various PRs, but don't know if there's some good reasons here.

ccache installation

Why is ccache installed in the runners? The cache file isn't persistent, so it seems like an unnecessary step.

See the [tool.cibuildwheel.*] sections of pyproject.toml.

* The Windows runners already have strawberry ccache installed anyway, so it's not needed there.

sccache w/ GH Action Cache

One neat optimization is to use sccache w/ the GH Action Cache. The GHA Cache is limited to 10GB.

I have a workflow using sccache for my primary dev builds, but 10GB isn't enough for the entire matrix. You could use it to speed up the most common smoke test builds.

Ninja on Windows

Why isn't Ninja enabled for Windows builds? Significantly speeds up the Windows builds. There's a couple of settings involved here, happy to open a PR... but not sure if there's some reason.

jemalloc on Windows

Any particular reason jemalloc is enabled on Windows debug builds?

On Windows, jemalloc is disabled for Windows Release, but enabled for Windows Debug builds. This fails locally for me, so I modified my local duckdb_loader.cmake to exclude when if(CMAKE_SYSTEM_NAME STREQUAL "Windows").

Noisy "uv export"

Would suggest adding "--quiet" to the uv export step in packaging_wheels. The uv lock output is very noisy.

Local Builds

A few tips I found for getting local builds running:

• Pass -vv to uv sync --no-build-isolation -vv step. This shows the build steps.
• edit: Adding --reinstall helped for rebuilds. My rebuild is usually: uv sync --active --no-build-isolation -vv --reinstall.
• I found re-enabled Unity helped significantly with my build times (DISABLE_UNITY='0'), ymmv.
• edit: Setting the external/duckdb to Unity enabled but the local project to disabled in CMakeLists.txt was my best balance of cache + build times, since I'm not touching anything inside external.
• Enable Ninja for Windows builds (see above)
• When working with multiple Python builds, was easier to use separate build dirs for each Python version. Could probably work this into pyproject.toml.
• If incrementally building with cmake, make sure to link or copy the _duckdb*.so file from the build dir to the location in the venv.
• sccache was not happy with the way DuckDB detects it... you end up with "sccache sccache compiler ...". Solution was:

cmake.define.CMAKE_C_COMPILER_LAUNCHER=""
cmake.define.CMAKE_CXX_COMPILER_LAUNCHER=""

Pytest Plugins

Some suggested plugins:

• pytest-xdist: This runs tests in parallel. I've run a few times with -n auto on my workflow... cuts test time significantly.
• pytest-randomly: This randomizes the order of tests, surfacing any test order dependencies or assumptions.
• pytest-timestamper: Adds timestamps to the verbose output. Helpful when wondering how long a test took.

[paultiq]

Some more comments on the updated/new Pre-commits

mypy

duckdb-python/.pre-commit-config.yaml

Line 37 in df7789c

entry: mypy -v

mypy -v is very verbose.

I suggest removing the -v, otherwise the output is flooded with LOG messages.

post-checkout-submodules

The pre-commit now calls git submodule update --init --recursive on every pre-commit run. This bumps the duckdb submodule frequently.

This is an odd and problematic thing to do in a pre-commit for a few reasons:

• It's changing the local environment as the last step of a pre-commit. After I've built and tested/etc.
• It means that PR ends up targeting a different duckdb submodule version... are we expected to commit the duckdb external that we used?
• It invalidates my local build cache (potentially) every time I run pre-commit. This can lead to a lot of rebuild time when we're just trying to make the linter happy.

Could this be changed to merely a WARNING? Doesn't seem like feature PRs should bump the submodule, nor that it should be changed as the last step of a pre-commit.

[paultiq]

Those were two separate points (mypy and submodule init).

Regarding mypy: My point is just that mypy -v is overly verbose and not meant for "regular" pre-commit use.

Example:

I inserted an error into duckdb/__init__.py (I changed __formatted_python_version__ to __formatted_python_bad_version__

pre-commit run mypy --all-files generates 541 lines of output.
After removing -v, pre-commit run mypy duckdb/__init__.py generates 11 lines of output... what I expect to see.

* My normal workflow is to use pre-commit run --all-files (in most projects) before PRs as a sanity check.
** --all-files is also what the code-quality workflow uses, so locally we should check against it.

Example of the regular output... what I want

duckdb/__init__.py:12: error: Module "_duckdb" has no attribute
"__formatted_python_bad_version__"; maybe "__formatted_python_version__"? 
[attr-defined]
    from _duckdb import (
    ^
Found 1 error in 1 file (checked 1 source file)

Example of the verbose output

LOG:  Mypy Version:           1.18.2
LOG:  Config File:            /home/ec2-user/git/duckdb-pythonf/pyproject.toml
LOG:  Configured Executable:  /home/ec2-user/git/duckdb-pythonf/.venv_3.13/bin/python3
LOG:  Current Executable:     /home/ec2-user/git/duckdb-pythonf/.venv_3.13/bin/python3
LOG:  Cache Dir:              .mypy_cache
LOG:  Compiled:               True
LOG:  Exclude:                ['duckdb/experimental/', 'duckdb/query_graph/', 'tests', 'scripts']
LOG:  Found source:           BuildSource(path='duckdb/__init__.py', module='duckdb', has_text=False, base_dir='/home/ec2-user/git/duckdb-pythonf', followed=False)
LOG:  Could not load cache for duckdb: duckdb/__init__.meta.json
LOG:  Metadata not found for duckdb
LOG:  Parsing duckdb/__init__.py (duckdb)
LOG:  Could not load cache for duckdb.value.constant: duckdb/value/constant/__init__.meta.json
LOG:  Metadata not found for duckdb.value.constant
LOG:  Parsing /home/ec2-user/git/duckdb-pythonf/duckdb/value/constant/__init__.py (duckdb.value.constant)
LOG:  Could not load cache for duckdb._dbapi_type_object: duckdb/_dbapi_type_object.meta.json
LOG:  Metadata not found for duckdb._dbapi_type_object
LOG:  Parsing /home/ec2-user/git/duckdb-pythonf/duckdb/_dbapi_type_object.py (duckdb._dbapi_type_object)
LOG:  Could not load cache for duckdb._version: duckdb/_version.meta.json
LOG:  Metadata not found for duckdb._version
LOG:  Parsing /home/ec2-user/git/duckdb-pythonf/duckdb/_version.py (duckdb._version)
LOG:  Could not load cache for _duckdb: _duckdb/__init__.meta.json
LOG:  Metadata not found for _duckdb
LOG:  Parsing /home/ec2-user/git/duckdb-pythonf/_duckdb-stubs/__init__.pyi (_duckdb)
LOG:  Metadata fresh for builtins: file /home/ec2-user/git/duckdb-pythonf/.venv_3.13/lib/python3.13/site-packages/mypy/typeshed/stdlib/builtins.pyi
LOG:  Metadata fresh for duckdb.value: file /home/ec2-user/git/duckdb-pythonf/duckdb/value/__init__.py
LOG:  Could not load cache for duckdb.sqltypes: duckdb/sqltypes/__init__.meta.json
...
LOG:  Metadata fresh for torch.distributed.elastic.multiprocessing.errors.handlers: file /home/ec2-user/git/duckdb-pythonf/.venv_3.13/lib/python3.13/site-packages/torch/distributed/elastic/multiprocessing/errors/handlers.py
LOG:  Metadata fresh for torch.distributed.elastic.multiprocessing.subprocess_handler.handlers: file /home/ec2-user/git/duckdb-pythonf/.venv_3.13/lib/python3.13/site-packages/torch/distributed/elastic/multiprocessing/subprocess_handler/handlers.py
LOG:  Metadata fresh for torch.distributed.elastic.multiprocessing.subprocess_handler.subprocess_handler: file /home/ec2-user/git/duckdb-pythonf/.venv_3.13/lib/python3.13/site-packages/torch/distributed/elastic/multiprocessing/subprocess_handler/subprocess_handler.py
LOG:  Metadata fresh for torch.distributed.elastic.timer.debug_info_logging: file /home/ec2-user/git/duckdb-pythonf/.venv_3.13/lib/python3.13/site-packages/torch/distributed/elastic/timer/debug_info_logging.py
LOG:  Metadata fresh for faulthandler: file /home/ec2-user/git/duckdb-pythonf/.venv_3.13/lib/python3.13/site-packages/mypy/typeshed/stdlib/faulthandler.pyi
LOG:  Loaded graph with 2016 nodes (0.480 sec)
LOG:  Found 446 SCCs; largest has 1167 nodes
LOG:  Processing 444 queued fresh SCCs
LOG:  Processing SCC of size 7 (_duckdb duckdb._version _duckdb._sqltypes duckdb._dbapi_type_object duckdb.sqltypes duckdb.value.constant duckdb) as inherently stale
LOG:  Deleting _duckdb /home/ec2-user/git/duckdb-pythonf/_duckdb-stubs/__init__.pyi _duckdb/__init__.meta.json _duckdb/__init__.data.json
LOG:  Deleting duckdb._version /home/ec2-user/git/duckdb-pythonf/duckdb/_version.py duckdb/_version.meta.json duckdb/_version.data.json
LOG:  Deleting _duckdb._sqltypes /home/ec2-user/git/duckdb-pythonf/_duckdb-stubs/_sqltypes.pyi _duckdb/_sqltypes.meta.json _duckdb/_sqltypes.data.json
LOG:  Deleting duckdb._dbapi_type_object /home/ec2-user/git/duckdb-pythonf/duckdb/_dbapi_type_object.py duckdb/_dbapi_type_object.meta.json duckdb/_dbapi_type_object.data.json
LOG:  Deleting duckdb.sqltypes /home/ec2-user/git/duckdb-pythonf/duckdb/sqltypes/__init__.py duckdb/sqltypes/__init__.meta.json duckdb/sqltypes/__init__.data.json
LOG:  Deleting duckdb.value.constant /home/ec2-user/git/duckdb-pythonf/duckdb/value/constant/__init__.py duckdb/value/constant/__init__.meta.json duckdb/value/constant/__init__.data.json
duckdb/__init__.py:12: error: Module "_duckdb" has no attribute "__formatted_python_bad_version__"; maybe "__formatted_python_version__"?  [attr-defined]
    from _duckdb import (
    ^
LOG:  Deleting duckdb duckdb/__init__.py duckdb/__init__.meta.json duckdb/__init__.data.json
LOG:  1 fresh SCCs (1 nodes) left in queue (and will remain unprocessed)
LOG:  Build finished in 2.630 seconds with 2015 modules, and 4 errors
Found 1 error in 1 file (checked 1 source file)

[paultiq]

Not sure what you see happening, can you include output? Note that:

• the submodule hook should only run post-checkout, and only if you’ve installed it. This should in no way change your env as part of a pre-commit hook.
• the submodule hook performs a checkout of the pinned commit in the parent project, it definitely shouldn’t bump. ‘checkout’ is the default behavior of ‘update’, and that means “the commit recorded in the superproject will be checked out in the submodule on a detached HEAD” (says man page) which is exactly what we want. I’m using this mostly because I switch between main and bugfix branches a lot, and this makes sure the submodule is always at the state that the parent project expects.
• you can disable the hooks if you have a lot of mypy errors while you fix them. That’s what I do myself, ‘uv run mypy|ruff’, etc.

It strikes me as incorrect for pre-commit to do a checkout (ever), as this changes my local repository unexpectedly and without a direct git action/decision. IMO, pre-commits are not intended for local repo maintenance actions... they're for "checks".

The instructions say to git config --local core.hooksPath .githooks/, which already handles this.

I'd suggest changing this to warn or fail, rather than checkout.

* I may be misunderstanding something here about when it runs, but githooks, checkouts, etc aren't things I expect to see in precommits. I'm imagining scenarios where the sub module is updated but I haven't updated the repo, because I'm still working on something... then I end up with mismatched sub module and repo

[evertlammerts]

Ah, the -v of the mypy config is probably just a leftover from debugging.

Re pre-commit, it sounds like git submodule update --init --recursive wasn't the culprit of your original problem. As for the more principled question: for all intents and purposes, git config --local core.hooksPath .githooks/ has the exact same effect as pre-commit install --hook-type post-checkout (which is no longer part of the docs, and won't be on main either after the next merge with the bugfix branch). Pre-commit is just a convenient way to add git hooks, of which there are many. It's not "incorrect" or "correct" to add hooks one way, another way, or not at all. Your workflow is yours. Submodules will always require extra care, especially in situations like ours where the main and bugfix branches significantly differ.

[paultiq]

Pre-commit is just a convenient way to add git hooks,

Perhaps the thing that I don't like here is that the instructions specifically say to register to the hook. But, I didn't want my working tree modified by default on a checkout.

Looking at this, I think where this surprised me was when doing work in one branch, switched to another branch and switched back. And, my original branch was no longer the same as when I started.

The problem I have here is:

• (documentation) the contributing instructions give a specific hook registration step, so I was surprised by the new pre commit
• (procedurally) moving submodule on checkout isn't something I would expect or want. A warning yes, but I don't want to end up in inconsistent states (where the submodule is updated but branch isn't)

That said, I can adapt now that I know this is in place. I'm just registering my surprise: at least a contributing doc update would be suggested.

[evertlammerts]

Docs have been updated last week. The README and CONTRIBUTING guides as well in the v1.4-andium branch, which will merge with main. If you have the hook enabled, do work on one branch, switch to another and then switch back, then the submodule should point at the correct sha, even when both branches point to different sha's. But: this only works if you've committed the submodule state. On the other hand, if you haven't and you do switch branches, git will complain.

[sakgoyal]

I was getting stuck in an infinite loop for some reason when doing pip install duckdb

Used the dev branch to fix it for py3.14 on windows.

[paultiq]

Did you follow the procedure in https://duckdb.org/docs/stable/dev/building/python ?

I'd suggest adding "-v" to the uv commands, so you can see what's happening.

[sakgoyal]

I was using pip not uv. I have uv installed, but I just did pip out of habit

[evertlammerts]

@sakgoyal for bugs please file an issue with logs and other details. It's hard to say what happens and whether it has anything to do with duckdb without some more context.

[sakgoyal]

ah. I used --verbose. seems like a compiled build was not available, so it started compiling the C code during install. I guess I didnt wait long enough for it to finish.

Output log:

C:\Users\Saksham>pip uninstall duckdb
Found existing installation: duckdb 1.5.0.dev56
Uninstalling duckdb-1.5.0.dev56:
  Would remove:
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\_duckdb-stubs\__init__.pyi
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\_duckdb-stubs\_func.pyi
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\_duckdb-stubs\_sqltypes.pyi
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\_duckdb.cp314-win_amd64.pyd
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\adbc_driver_duckdb\*
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\duckdb-1.5.0.dev56.dist-info\*
    c:\users\saksham\appdata\local\programs\python\python314\lib\site-packages\duckdb\*
Proceed (Y/n)? y
  Successfully uninstalled duckdb-1.5.0.dev56

C:\Users\Saksham>pip install --verbose duckdb
Using pip 25.2 from C:\Users\Saksham\AppData\Local\Programs\Python\Python314\Lib\site-packages\pip (python 3.14)
Collecting duckdb
  Using cached duckdb-1.4.1.tar.gz (18.5 MB)
  Running command pip subprocess to install build dependencies
  Using pip 25.2 from C:\Users\Saksham\AppData\Local\Programs\Python\Python314\Lib\site-packages\pip (python 3.14)
  Collecting scikit-build-core>=0.11.4
    Obtaining dependency information for scikit-build-core>=0.11.4 from https://files.pythonhosted.org/packages/43/49/ec16b3db6893db788ae35f98506ff5a9c25dca7eb18cc38ada8a4c1dc944/scikit_build_core-0.11.6-py3-none-any.whl.metadata
    Using cached scikit_build_core-0.11.6-py3-none-any.whl.metadata (18 kB)
  Collecting pybind11>=2.6.0 (from pybind11[global]>=2.6.0)
    Obtaining dependency information for pybind11>=2.6.0 from https://files.pythonhosted.org/packages/cd/8a/37362fc2b949d5f733a8b0f2ff51ba423914cabefe69f1d1b6aab710f5fe/pybind11-3.0.1-py3-none-any.whl.metadata
    Using cached pybind11-3.0.1-py3-none-any.whl.metadata (10.0 kB)
  Collecting setuptools_scm>=8.0
    Obtaining dependency information for setuptools_scm>=8.0 from https://files.pythonhosted.org/packages/11/cc/5fda7ed3aab0814a86e428846b6b386f55093f4621d86e3cd081ae126c27/setuptools_scm-9.2.1-py3-none-any.whl.metadata
    Using cached setuptools_scm-9.2.1-py3-none-any.whl.metadata (7.7 kB)
  Collecting packaging>=23.2 (from scikit-build-core>=0.11.4)
    Obtaining dependency information for packaging>=23.2 from https://files.pythonhosted.org/packages/20/12/38679034af332785aac8774540895e234f4d07f7545804097de4b666afd8/packaging-25.0-py3-none-any.whl.metadata
    Using cached packaging-25.0-py3-none-any.whl.metadata (3.3 kB)
  Collecting pathspec>=0.10.1 (from scikit-build-core>=0.11.4)
    Obtaining dependency information for pathspec>=0.10.1 from https://files.pythonhosted.org/packages/cc/20/ff623b09d963f88bfde16306a54e12ee5ea43e9b597108672ff3a408aad6/pathspec-0.12.1-py3-none-any.whl.metadata
    Using cached pathspec-0.12.1-py3-none-any.whl.metadata (21 kB)
  Collecting setuptools (from setuptools_scm>=8.0)
    Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/a3/dc/17031897dae0efacfea57dfd3a82fdd2a2aeb58e0ff71b77b87e44edc772/setuptools-80.9.0-py3-none-any.whl.metadata
    Using cached setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
  Collecting pybind11-global==3.0.1 (from pybind11[global]>=2.6.0)
    Obtaining dependency information for pybind11-global==3.0.1 from https://files.pythonhosted.org/packages/3f/62/a2e1f90a2af23a768371cdfd0fc73a5e2e75ccb2da1f2c67b421c300e9ba/pybind11_global-3.0.1-py3-none-any.whl.metadata
    Using cached pybind11_global-3.0.1-py3-none-any.whl.metadata (9.9 kB)
  Using cached scikit_build_core-0.11.6-py3-none-any.whl (185 kB)
  Using cached pybind11-3.0.1-py3-none-any.whl (293 kB)
  Using cached setuptools_scm-9.2.1-py3-none-any.whl (62 kB)
  Using cached packaging-25.0-py3-none-any.whl (66 kB)
  Using cached pathspec-0.12.1-py3-none-any.whl (31 kB)
  Using cached pybind11_global-3.0.1-py3-none-any.whl (539 kB)
  Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
  Installing collected packages: setuptools, pybind11-global, pybind11, pathspec, packaging, setuptools_scm, scikit-build-core
    Creating C:\Users\Saksham\AppData\Local\Temp\pip-build-env-27laoc6u\overlay\Scripts

  Successfully installed packaging-25.0 pathspec-0.12.1 pybind11-3.0.1 pybind11-global-3.0.1 scikit-build-core-0.11.6 setuptools-80.9.0 setuptools_scm-9.2.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command pip subprocess to install backend dependencies
  Using pip 25.2 from C:\Users\Saksham\AppData\Local\Programs\Python\Python314\Lib\site-packages\pip (python 3.14)
  Collecting cmake>=3.29.0
    Obtaining dependency information for cmake>=3.29.0 from https://files.pythonhosted.org/packages/7c/d0/73cae88d8c25973f2465d5a4457264f95617c16ad321824ed4c243734511/cmake-4.1.0-py3-none-win_amd64.whl.metadata
    Using cached cmake-4.1.0-py3-none-win_amd64.whl.metadata (6.5 kB)
  Using cached cmake-4.1.0-py3-none-win_amd64.whl (37.6 MB)
  Installing collected packages: cmake
    Creating C:\Users\Saksham\AppData\Local\Temp\pip-build-env-27laoc6u\normal\Scripts
  Successfully installed cmake-4.1.0
  Installing backend dependencies ... done
  Running command Preparing metadata (pyproject.toml)
  *** scikit-build-core 0.11.6 using CMake 4.1.0 (metadata_wheel)
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: duckdb
  Running command Building wheel for duckdb (pyproject.toml)
  [duckdb_pytooling.build_backend] Building duckdb wheel from sdist. Reading duckdb version from file.
  [duckdb_pytooling.build_backend] cmake.define.OVERRIDE_GIT_DESCRIBE set to v1.4.1
  *** scikit-build-core 0.11.6 using CMake 4.1.0 (wheel)
  *** Configuring CMake...
  loading initial cache file C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\CMakeInit.txt
  -- Building for: Visual Studio 17 2022
  -- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.19045.
  -- The CXX compiler identification is MSVC 19.44.35217.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Using C:/Users/Saksham/AppData/Local/Microsoft/WinGet/Links/ccache.exe as C compiler launcher
  -- Using C:/Users/Saksham/AppData/Local/Microsoft/WinGet/Links/ccache.exe as C++ compiler launcher
  -- Found Python: C:/Users/Saksham/AppData/Local/Programs/Python/Python314/python.exe (found suitable version "3.14.0", minimum required is "3.8") found components: Interpreter Development.Module Development.Embed
  -- Performing Test HAS_MSVC_GL_LTCG
  -- Performing Test HAS_MSVC_GL_LTCG - Success
  -- Found pybind11: C:/Users/Saksham/AppData/Local/Temp/pip-build-env-27laoc6u/overlay/Lib/site-packages/pybind11/include (found version "3.0.1")
  -- DuckDB: Configured for release build
  CMake Warning at cmake/duckdb_loader.cmake:131 (message):
    jemalloc extension is only supported on Windows in Debug builds.

    Removing jemalloc from extension list.

    In non-debug builds, jemalloc is only supported on Linux.
  Call Stack (most recent call first):
    cmake/duckdb_loader.cmake:251 (_duckdb_validate_jemalloc_config)
    CMakeLists.txt:47 (duckdb_add_library)


  -- DuckDB Configuration:
  --   Source: C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb
  --   Build Type:
  --   Native Arch: OFF
  --   Unity Build Disabled: OFF
  --   Extensions: core_functions;json;parquet;icu
  -- Found Python3: C:/Users/Saksham/AppData/Local/Programs/Python/Python314/python.exe (found version "3.14.0") found components: Interpreter
  -- The C compiler identification is MSVC 19.44.35217.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - not found
  -- Found Threads: TRUE
  -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.49.0.windows.1")
  fatal: not a git repository (or any of the parent directories): .git
  CMake Warning at external/duckdb/CMakeLists.txt:306 (message):
    git is available (at C:/Program Files/Git/cmd/git.exe) but has failed to
    execute 'log -1 --format=%h'.  Consider providing explicit GIT_COMMIT_HASH


  -- GIT_COMMIT_HASH has length 0 different than the expected 10
  -- git hash , version v1.4.1, extension folder v1.4.1
  -- Extensions will be deployed to: C:/Users/Saksham/AppData/Local/Temp/tmpwqqve7ja/build/repository
  CMake Deprecation Warning at external/duckdb/CMakeLists.txt:1270 (message):
    CORE_EXTENSIONS is deprecated.  Use BUILD_EXTENSIONS instead.


  -- Load extension 'core_functions' from 'C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/extensions' @ v1.4.1
  -- Load extension 'json' from 'C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/extensions' @ v1.4.1
  -- Load extension 'parquet' from 'C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/extensions' @ v1.4.1
  -- Load extension 'icu' from 'C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/extensions' @ v1.4.1
  -- Extensions linked into DuckDB: [core_functions, json, parquet, icu]
  -- Configuring done (28.0s)
  -- Generating done (5.0s)
  -- Build files have been written to: C:/Users/Saksham/AppData/Local/Temp/tmpwqqve7ja/build
  *** Building project with Visual Studio 17 2022...
  MSBuild version 17.14.23+b0019275e for .NET Framework

  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(548,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\ZERO_CHECK.vcxproj]
    1>Checking Build System
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(548,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\fastpforlib\duckdb_fastpforlib.vcxproj]
    Building Custom Rule C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/third_party/fastpforlib/CMakeLists.txt
    bitpacking.cpp
    duckdb_fastpforlib.vcxproj -> C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\fastpforlib\Release\duckdb_fastpforlib.lib
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(548,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\fmt\duckdb_fmt.vcxproj]
    Building Custom Rule C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/third_party/fmt/CMakeLists.txt
    format.cc
    duckdb_fmt.vcxproj -> C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\fmt\Release\duckdb_fmt.lib
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(548,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\fsst\duckdb_fsst.vcxproj]
    Building Custom Rule C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/third_party/fsst/CMakeLists.txt
    libfsst.cpp
    duckdb_fsst.vcxproj -> C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\fsst\Release\duckdb_fsst.lib
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(548,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\hyperloglog\duckdb_hyperloglog.vcxproj]
    Building Custom Rule C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/third_party/hyperloglog/CMakeLists.txt
    hyperloglog.cpp
    sds.cpp
    Generating Code...
    duckdb_hyperloglog.vcxproj -> C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\hyperloglog\Release\duckdb_hyperloglog.lib
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(548,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\Saksham\AppData\Local\Temp\tmpwqqve7ja\build\duckdb\third_party\mbedtls\duckdb_mbedtls.vcxproj]
    Building Custom Rule C:/Users/Saksham/AppData/Local/Temp/pip-install-lbbitu_r/duckdb_dbbb1e379bbc4809a7c6050f15f65235/external/duckdb/third_party/mbedtls/CMakeLists.txt
    mbedtls_wrapper.cpp
    aes.cpp
    
/// it's still going at the moment

[paultiq]

Yah, the windows builds are painfully slow / single threaded by default.

I enabled ninja (#77) or you can enable msbuild's multiprocessing. And, I use sccache/ccache to speed up rebuilds.

* fyi, that output shows using pip and not the process described here: https://duckdb.org/docs/stable/dev/building/python#2-install-dependencies-and-build . I don't know if that'll lead to other problems.