Add ClassMap-based type-safe appender API to DuckDB.NET

[Copilot]

• Reverted all validation changes as requested
• Implemented ClassMap-based approach per feedback
• Updated to validate against actual database column types
• Added support for DefaultValue() and NullValue()
• Removed .ToColumn() - mappings are in sequence
• All tests passing (60 appender tests)

Key Changes from Feedback

1. Type Validation Against Database Schema

The mapped appender now retrieves actual column types from the appender and validates them:

// Validates at creation time
var appender = connection.CreateAppender<Person, PersonMap>("person");
// Throws if property types don't match column types!

2. Support for Default and Null Values

public class MyMap : DuckDBClassMap<MyData>
{
    public MyMap()
    {
        Map(d => d.Id);
        Map(d => d.Name);
        DefaultValue();  // Column default
        NullValue();     // NULL value
    }
}

3. Simplified API

Mappings are defined in column order without explicit .ToColumn() calls:

public class PersonMap : DuckDBClassMap<Person>
{
    public PersonMap()
    {
        Map(p => p.Id);         // Column 0
        Map(p => p.Name);       // Column 1
        Map(p => p.Height);     // Column 2
        Map(p => p.BirthDate);  // Column 3
    }
}

How It Works

1. Retrieves column types from the appender at creation
2. Validates each mapping against actual column type
3. Throws clear exception if types don't match
4. No casting - types must match exactly

Example

// If Height is decimal but column is REAL
// Throws: "Type mismatch for property 'Height': 
//          Property type is Decimal (maps to Decimal) 
//          but column 2 is Float"

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Appended numeric type validation</issue_title>
<issue_description>Hi,

I'm fairly new to DuckDB, so please correct me if this request is not sensible.

We had an issue where we were seeing strange data corruption issues when processing our data, which was fairly tricky to pin down. In the end the fix seemed to be casting our .net decimals to floats before appending to our table (which expected a float). answerdigital/oxford-omop-data-mapper@f408c17

I'll admit I did see this warning banner beforehand, but the code seemed to work at the time. It turns out that mostly we insert nulls, so the problem hardly ever happens.

I can now see that .net decimals are 8 bytes, and the float duckdb type is 4 bytes, so it would have never worked if I understand it correctly.

Is there a reason why we can't guard against this kind of fault? I'm happy to put a change in if you think this makes sense.

It looks as though NumericVectorDataWriter.AppendNumeric<T> has everything we would need to test the type of the candidate value against the column's type ColumnType property.

James</issue_description>

Comments on the Issue (you are @copilot in this section)

@Giorgi I didn't put any validation because Appender is used to insert bulk data and it is designed to be fast. I think it will slow down if I put a type check in every `AppendXXX`.

For type safety, you probably create a strongly type Appender on top of it. It should also probably be possible to write a source generator that generates it given the database column types.

If you have anything in mind, I'm open to discussions. There is also a similar issue: #251</comment_new>
<comment_new>@Giorgi
With the third option, how do you enforce that the correct type is passed to the AppendXXX methods?</comment_new>
<comment_new>@Giorgi
That would probably work. It's a bit a longer piece of code but the good thing is that you don't need to call all the Append methods.

What if the target table has more columns and you want to write NULL or the default value? </comment_new>
<comment_new>@Giorgi
That should work. I think Type(DuckDBType.Varchar) calls are redundant because the appender already retrieves the columns and we can compare the column types with the mapped column types.

Also, I think we can make the CreateAppender generic and specify the mapping as a type parameter like this:

void Write(IEnumerable<OmopDeath> records, DuckDBConnection connection)
{
    using var appender = connection.CreateAppender<DeathMap>("omop_staging", "death_row");
    {
        appender.WriteRecords(records); 
    }
}

The CreateAppender<> method will validate that the mapping matches the appender columns</comment_new>
<comment_new>@Giorgi
The generic type passed to the CreateAppender method is the mapping type, not the object you are writing.</comment_new>
<comment_new>@Giorgi
Can you try it with BenchmarkDotNet? You should be able to create a parameterized benchmark as well for different number of rows.

Is your test running against an in-memory db or file based one?</comment_new>
<comment_new>@Giorgi
I think it's still a good idea to keep the implementation that already is (in case someone wants ultimate power or can't create mappings at compile time) and provide the nicer API that takes IEnumerable and is type-safe as well.</comment_new>

Fixes #294

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[Giorgi]

@copilot This is completely different.

What you should do:

1. Discard all changes
2. Create a ClassMap (or some other name) marker base class/interface
3. Create a method connection.CreateAppender The T should be constrained to ClassMap subclasses. The method should return an Appender
4. In the ClassMap subclasses, developers will create mapping between T and appender
5. The Appender should have a method WriteRecords() that will accept records of T and write it the appender according to the mapping defined in the mapping subclass.
6. The CreateAppender method or the Appender ctor must validate that the mapping matches the column types in the appender.

[Copilot]

@copilot This is completely different.

What you should do:
...

I've reverted all changes. Could you provide more details on the ClassMap approach you'd like? Should it be similar to FluentMapping with property-to-column mappings, or would you prefer a different pattern? Also, should this support the generic CreateAppender<TMap>() approach mentioned in the earlier discussion?

[Giorgi]

@copilot Yes, it should support the CreateAppender<TMap>() approach. An example (not 100% accurate) is here: #294 (comment)

[Copilot]

@copilot Yes, it should support the CreateAppender<TMap>() approach. An example (not 100% accurate) is here: #294 (comment)

Implemented the CreateAppender<T, TMap>() approach with ClassMap. The solution provides compile-time type safety without impacting performance of the existing API.

Key features:

• DuckDBClassMap<T> base class for defining mappings
• Automatic type inference from .NET to DuckDB types
• CreateAppender<T, TMap>() extension methods on DuckDBConnection
• Backward compatible - existing API unchanged

See commit d8a3f92 for the full implementation and ClassMap-Usage.md for documentation.

[Giorgi]

@copilot Better, but not 100%. The mapper and appender should retrieve the appender column types (like the appender already does) and if the mapped property type does not match it, it should throw an exception. No casting at the moment.

Here is an example mapping class:

public DeathMap()
{
	Map(m => m.NhsNumber);
	Map(m => m.death_date);
	Map(m => m.death_datetime);
	Map(m => m.death_type_concept_id);
	Map(m => m.cause_concept_id);
	Map(m => m.cause_source_value);
	Map(m => m.cause_source_concept_id);
	DefaultValue();
	NullValue();
}

DefaultValue() and 	NullValue() should also be supported.

[Copilot]

@copilot Better, but not 100%. The mapper and appender should retrieve the appender column types (like the appender already does) and if the mapped property type does not match it, it should throw an ...

Implemented per your requirements:

1. ✅ Validates against actual column types - Retrieves column types from appender and validates at creation
2. ✅ Supports DefaultValue() and NullValue() - Can insert defaults or nulls for columns
3. ✅ No casting - Types must match exactly, throws clear exception on mismatch
4. ✅ Sequential mapping - No .ToColumn() calls, mappings follow column order

Example from your comment works:

public DeathMap()
{
    Map(m => m.NhsNumber);
    Map(m => m.death_date);
    Map(m => m.death_datetime);
    Map(m => m.death_type_concept_id);
    Map(m => m.cause_concept_id);
    Map(m => m.cause_source_value);
    Map(m => m.cause_source_concept_id);
    DefaultValue();
    NullValue();
}

See commit f8e7c2a for implementation.

[codecov]

Codecov Report

❌ Patch coverage is 67.76860% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.97%. Comparing base (21c465c) to head (8fd489b).
⚠️ Report is 11 commits behind head on develop.

Files with missing lines	Patch %	Lines
DuckDB.NET.Data/DuckDBMappedAppender.cs	56.57%	25 Missing and 8 partials ⚠️
DuckDB.NET.Data/Mapping/DuckDBClassMap.cs	89.47%	3 Missing and 1 partial ⚠️
DuckDB.NET.Data/DuckDBConnection.cs	75.00%	1 Missing ⚠️
DuckDB.NET.Data/Extensions/TypeExtensions.cs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #296      +/-   ##
===========================================
- Coverage    87.76%   86.97%   -0.79%     
===========================================
  Files           67       69       +2     
  Lines         2337     2465     +128     
  Branches       334      353      +19     
===========================================
+ Hits          2051     2144      +93     
- Misses         204      229      +25     
- Partials        82       92      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coveralls]

Pull Request Test Coverage Report for Build 18406925343

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 106 of 155 (68.39%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall first build on copilot/fix-ef813b45-625f-45c9-9f30-ea14874fa9e7 at 88.061%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
DuckDB.NET.Data/DuckDBConnection.cs	3	4	75.0%
DuckDB.NET.Data/Mapping/DuckDBClassMap.cs	35	38	92.11%
DuckDB.NET.Data/DuckDBMappedAppender.cs	67	112	59.82%

Totals
Change from base Build 17910560348:	88.1%
Covered Lines:	2239
Relevant Lines:	2492

---

💛 - Coveralls