Skip to content

wof emit fails to handle alt files (SPR) #37

Open

Description

@thisisaaronland
> sh ./fetch2.sh
2025/09/18 10:26:46 ERROR Failed to write record %s (%s), %w path=data/856/333/45/85633345-alt-quattroshapes.geojson count=1 id=85633345 "alt geom"=true rel_path=856/333/45/85633345-alt-quattroshapes.geojson 856/333/45/85633345-alt-quattroshapes.geojson=data/856/333/45/85633345-alt-quattroshapes.geojson error="Failed to derive SPR from 856/333/45/85633345-alt-quattroshapes.geojson, Failed to derive ID, 'wof:id' property not found"
2025/09/18 10:26:46 Failed to run 'emit' command, Failed to write record for 856/333/45/85633345-alt-quattroshapes.geojson, Failed to derive SPR from 856/333/45/85633345-alt-quattroshapes.geojson, Failed to derive ID, 'wof:id' property not found

Activity

  1. thisisaaronland commented on Sep 18, 2025

    @thisisaaronland
    MemberAuthor

    This looks like a go-writer-geoparquet issue:

    > git grep 'Failed to derive SPR from'
    vendor/github.com/whosonfirst/go-writer-geoparquet/v3/geoparquet.go:                    return 0, fmt.Errorf("Failed to derive SPR from %s, %w", key, err)
    
  2. thisisaaronland commented on Sep 18, 2025

    @thisisaaronland
    MemberAuthor

    Actually, the problem seems to be in tidwall/sjson assigning the (WOF) SPR as initial properties. For example, in go-writer-geoparquet/geoparquet.go:

       186 |        wtf_body, err := json.Marshal(wof_spr)
       187 |
       188 |	if err != nil {
       189 |                return 0, err
       190 |	}
       191 |
       192 |        if strings.Contains(string(body), "WOFParentId"){
       193 |                slog.Info("WTF 0", "body", string(wtf_body))
       194 |	}
       195 |
       196 |	// START OF wrangle properties in to something GeoParquet can work with
       197 |
       198 |	old_props := gjson.GetBytes(body, "properties")
       199 |
       200 |	body, err = sjson.SetBytes(body, "properties", wof_spr)
       201 |
       202 |	if err != nil {
       203 |                return 0, fmt.Errorf("Failed to update properties for %s, %w", key, err)
       204 |        }
       205 |
       206 |	if strings.Contains(string(body), "WOFParentId"){
       207 |                slog.Info("WTF 1", "body", string(body))
       208 |        }
    

    WTF 0 is not triggered but WTF 1 is always triggered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingemit

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions