Stability of the ID field in FilterLists.json

[ignoramous]

Hi Colin,

We plan to use filter-lists for our content-blocking DoH project at rethinkdns.com/configure. Right now, we are reliant on NextDNS' lists but are growing concerned by the maintainer's reluctance to license it favourably for other FOSS projects.

We'd like to know if the IDs once assigned to a list would never be re-assigned?

For example: Malware Domain List is now effectively defunct; which has the ID of 10 (as of writing). Is there ever a possibility this ID gets recycled and used for another list?

We ask because it isn't uncommon at all for lists to die, but ideally when they do, our use-case is such that we'd not want to see those IDs reassigned to another list. If that's the case, fair and good. If not, that's fair too, but we'd like a confirmation either way.

Alternatively, when a list dies, does a new field indicate its dead status or is the list simply removed from the .json (with the ID never reassigned / open to reassignment)?

Thanks.

Ref: FilterList.json.

[collinbarrett]

Hey, cool app!

Yes, the plan is to never recycle IDs for any of the data types (.json files). I can't say that we have a technical solution to prevent that, but the handful of follks that contribute data updates know that policy, I think, and I watch for it on PRs as well. The system will certainly not let a duplicate ID be added, but if an ID is deleted, it's technically possible for it to be recycled, but we certainly try/plan not to.

We also have played with #713 for preserving data about lists that are taken offline. It's not currently implemented after a major re-write of the app, but we might add it back. I think soft delete for all of our data types could be ideal because there have been a couple of times where a list goes down for awhile and then reappears.

But, traditionally when we have wanted to remove data, we have deleted it.

[collinbarrett]

@ignoramous , just curious if your app is still using the FilterLists api? if so, do you know what endpoints specifically you use?

I'm doing some re-architecting that might involve some breaking API changes, so I want to understand the impact if any other actors are currently using the API besides the FilterLists website.

thanks.

[ignoramous]

Hi Colin,

Thanks for checking this with us! (:

Our rewrite to use FilterLists isn't in prod at this time, so we won't be affected by changes to either the JSON structure or its API guarantees.

Btw, would the newer arch still have immutable ID assignments?

[collinbarrett]

Ok, cool.

I'm not sure yet. I'm going to try not to change the current API signature unless I need to, but just trying to understand the impact if I do. But, most likely will be keeping the same immutable integer IDs for all of the resources.

[ignoramous]

It works either way for us. Don't be beholden to our requirements :) Thanks again, appreciate the transparency.