ephemeralForTest storage engine?

[AustinScola]

Hi! First some background: I used mongodb-memory-server for a NodeJs project and now I'm working on a Python project. There doesn't seem to be a good equivalent of mongo-memory-server for Python so I am making a very minimal version of one.

I noticed that mongodb-memory-server uses "ephemeralForTest" as the default storage engine. In the MongoDB docs I only see two options for storage engines? So I'm wondering where the option for "ephemeralForTest" is coming from?

[hasezoey]

its true that the current documentation for mongodb doesnt list ephemeralForTest anywhere anymore, and instead uses inMemory (i guess), but this does not work as an storage engine option (at least in 4.0 & 4.2)

i dont know if any earlier versions of the documentation is still available, but at some point it was referenced

but as you can see for those 2 options, it states for inMemory:

Available in MongoDB Enterprise only.

i am not quite sure what this storage engine does differently, but i think it basically is ephemeralForTest but also stores to disk and so allows authentication to work?

and ephemeralForTest is an in-memory engine, which the name already says "for test(s)", so it is not meant to be used as an actual production engine, but is completely fine for simple test environments

compared to other engines (like wiredTiger), ephemeralForTest has disadvantages, the main ones being (and that i know of):

• does not store anything to disk at any point (including shutdown)
• because of the above, authentication does not really work to its fullest
• it is probably not as optimized as other engines
• can only store as much as there is ram (and swap)

[AustinScola]

That is interesting. So it still works despite being absent from the documentation? I wonder if MongoDB is going to remove it at some point or if this was an accident on their part. Thanks for listing some of the disadvantages as well. I hadn't considered that it might be less optimized.

A couple follow up questions:

• Why would you want to store data on shutdown?
• Do you know why authentication requires disk to work properly?

[hasezoey]

Do you know why authentication requires disk to work properly?

because to enable auth, you have to start the instance without auth, create an admin account (and maybe additional accounts), and then restart with auth enabled, which ties into:

Why would you want to store data on shutdown?

basically to enable auth you have to restart an instance, and because it does not store anything, it can never be done (that is why the docs(this package) list that wiredTiger should be used when wanting authentication)

I wonder if MongoDB is going to remove it at some point or if this was an accident on their part

i think (hope) they dont remove it, because it speeds up tests really much compared to wiredTiger, and i think they also use it for their tests as well

[AustinScola]

I'm a bit curious why it requires restart. Seems like you could enable auth in software but I don't know much about how any of that works (as is evidenced by my questions). Thank you for the explanation.

[hasezoey]

i also dont quite know why, but on the time where i was implementing it for this package, it was only possible to enable / disable auth with the command-line argument --auth