I want to experiment with sharing bite-sized tidbits and stories of how we solve problems, from the simple to the more complex. A sort of behind-the-scenes or a “day in the life” of our team.
In this episode, Karol and I discuss how to best position the new files server in our infrastructure in a way that is scalable and reliable for us, while also maintaining a user-friendly self-hosting experience for users who self-host the backend infrastructure, but not the frontend applications. The goal is allowing users to self-host their own encrypted files server and use it with our official applications on desktop, web, and mobile.
Why is the files server not behind the API Gateway?
Hmm so the files server host is set via an ENV variable in the clients. This means that if I sign into a custom server using
app.standardnotes.com, I won't be able to upload files correctly, because the files server will be hardcoded to
files.standardnotes.com, but my account is on a custom server.
This means that a user will not be able to self-host the files server and use our apps at the same time. Because again the value they enter into Custom Sync Server when signing in does not apply to files.
My first guess as to why files is separated from the broad api-gateway infra is because files is resource intensive and routing a lot of file bytes through api-gateway may bog it down. If there is no way around this, then it seems that when a user signs in we will have to add another input field for the files server. But this UX stands to get confusing.
Any possible solutions?
What about putting the files server up at
api.standardnotes.com/files 🤔 So the client will just append
/files to the user’s existing server path.
My first guess is that this is because files is resource intensive and routing a lot of file bytes through api-gateway may bog it down.
Yes, that is precisely the reason. File operations will be high memory consuming ones and could potentially bring a container down until we have everything super tweaked. So that is a risk that we don't want to take. Essentially we are rendering it impossible for the files service to bring down the entire infrastructure.
This also means that a user will not be able to self-host the files server and use our apps at the same time. Because again the value they enter into Custom Sync Server when signing in does not apply to files.
Hmm yeah, but then again if they're self hosting the app they'll have to provide the env var upon starting. So, it's part of the setup that you already have to go through if you're self-hosting.
What about putting the files server up at
api.standardnotes.com/files🤔 So the client will just append
/filesto the user’s existing server path.
Technically possible but that would have to be a reroute on the load balancer so that the traffic would not hit the API Gateway. Also this is something that we will not be able to mimic on the self-hosting setup since we don't have a load balancer there. It's possible but creates a different set of issues altogether.
Regarding self hosting, most users self host the server and not the clients. So there is no opportunity for them to configure the env vars on the client to point to their files server.
In general though our clients are meant to be server agnostic regarding data endpoints and meant to be fully configurable from a UX perspective. So if your company for example is self hosting the SN infra and you want to use our mobile app from the App Store, we’d have to have an input field on sign in to specify the files server.
Any other possible solutions you could think of?
What if the api-gateway advertised its respective files server to clients in the meta response?
Hmm that would mean that the file server would have to somehow "register" with the api-gateway right? Something like sending an HTTP request upon container boot with "my host" to the "api gateway" url. Then we might do a model of ServerSettings just like we have UserSettings which could be retrieved by the client. So the app would fetch let's say
Hmm I was thinking more that the api-gateway would just have a
FILES_HOST env var, and then clients can just request that via either a new endpoint or piggybacking on the meta response included in all requests?
Hmm yeah, that could work too. Much easier to implement I think. I'll put something together.
And here’s the result of our conversation: https://github.com/standardnotes/api-gateway/commit/3bc84a8ca8c21241a20feae17d861370b26a7ae4
I hope you enjoyed this behind the scenes of how we tackle everyday problems. If interesting, we can definitely share more tales like this 🙂