Is there a best way to mount local storage from Web Worker?

[atty303]

I use ZenFS from an Emscripten app running on Web Worker. Is there a good way to reference LocalStorage in such a case?

I will write about an experiment that I have implemented and that has not yet worked well.
I created SimpleAsyncStore based on SimpleSyncStore.

https://github.com/atty303/pob-web/blob/8979474ecfaa8d953eff22f64638d2c4adc178a7/packages/driver/src/fs.ts#L490-L626

And it is mounted.

https://github.com/atty303/pob-web/blob/8979474ecfaa8d953eff22f64638d2c4adc178a7/packages/driver/src/worker.ts#L117-L126

GET/PUT/DELETE callbacks are forwarded to the main thread using Comlink.

https://github.com/atty303/pob-web/blob/8979474ecfaa8d953eff22f64638d2c4adc178a7/packages/driver/src/main.ts#L35-L63

It is difficult to explain, but FS is unstable. Files cannot be read immediately after worker startup, or an exception occurs in Sync after a file is written.

core/src/backends/AsyncStore.ts

Line 455 in ca98b1e

throw ErrnoError.With('ENOENT', resolve(parent, filename), '_findINode');

In particular, regarding this exception, we observed that AsyncStoreFS#sync seemed to be executed before AsyncStoreFS#commitNewFile was completed. If so, the parent's DirListing has not yet been PUT, but sync is getting it and this exception is being raised.

[atty303]

I also tried the Port backend.

const fs = await zenfs.resolveMountConfig({ backend: WebStorage });
zenfs.attachFS(this.worker, fs);

However, it failed with the following error. It seems to be trying to postMessage a local storage object.

fs.js:212 Uncaught (in promise) DOMException: Failed to execute 'postMessage' on 'Worker': Storage object could not be cloned.
    at handleRequest (http://localhost:5173/node_modules/.vite/deps/chunk-FUNVVB6Z.js?v=46be177c:9146:8)

[james-pre]

Hi @atty303!

Is there a good way to reference localStorage in such a case?

I would recommend using the Port backend if you are using a web worker.

I created SimpleAsyncStore based on SimpleSyncStore.

This is really cool! I'm actually reworking the system of stores right now to use a single. unified Store class, with async mirroring.

And it is mounted ...

I recommend you inline the config, since it can be easier to understand and eliminates otherwise used variables.

For your use case:

const rootZip = await fetch(`${assetPrefix}/root.zip`);

await zenfs.configure({
	mounts: {
		'/root': {
			backend: Zip,
			zipData: await rootZip.arrayBuffer(),
			name: 'root.zip',
		},
		'/user': {
			name: 'LocalStorage',
			backend: SimpleAsyncFS,
			store: new SimpleAsyncStore(getCallback, putCallback, removeCallback),
		},
	},
});

In particular, regarding this exception, we observed that AsyncStoreFS#sync seemed to be executed before AsyncStoreFS#commitNewFile was completed. If so, the parent's DirListing has not yet been PUT, but sync is getting it and this exception is being raised.

This seems like a very frustrating bug to solve... I will create an issue to track it.

However, it failed with the following error. It seems to be trying to postMessage a local storage object.

This was because FileSystem.ready returns a Promise<this>, meaning PortFS's ready would call the remote ready, which would return the Promise<this>, leading to an attempt to pass the Storage. core e877fdc changes FileSystem.ready to return a Promise<void> instead.

[atty303]

Okay, thank you, I'm happy to see the issue resolved.
I think ZenFS is a very well designed and useful library. Please keep up the good work!

[james-pre]

Thank you! If you find something with ZenFS that is difficult to use or experience any issues, please reach out and I will try to resolve it as quickly as possible.