Selfhosted

Yes no staging because it's something used at most by 2 concurrent users, we were ok with 95% reliability (we discovered it was disabled after at least two weeks lol)

Otherwise we would just have signed up to one of the many cloud forms sites at $100/year

Backups daily but it's unthinkable to revert something like nextcloud to a months old one

Subscribed to both newsletter and RSS feed to know about issues (the command to update the docker images isn't automated but manually issued). The maintainer of the forms app is nextcloud itself so any incompatibility should have been written in red bold characters in the blog posts and newsletter.

To be fair a certain security company was in global news for exactly that same send it behavior. Why waste precious resources on multiple instances? Investors hate waste. 😅

One that lacks a good IT department apparently

If I understand correctly, nextcloud automatically updated ... which I didn't think it would, normally. Maybe it's a "feature" of the AIO docker image?

Docker images should never self update - that's an anti pattern. They should be static code. The only time I would expect a docker image to "auto update" is if I was using the "latest" or "stable" tag and Compose/Kubernetes/I repull the image - but the image should never update itself.

Yes, OP bit off more than they could chew. Nextcloud, however, is breaking the entire purpose of Docker images by having an auto-updater at all.

They're releasing a new version every two month or so and dropping them rapidly from support, pinning it with a tag means that in 12 months the install would be exploitable.

Now, I did directly to production because this is low priority stuff, but it would have happened even with a testing stage. I would have never noticed that the forms apps was disabled, the system disabled it without any notification.

You would expect that an official app supports the latest release, no?

This wasn't an app released by a nobody in their free time, this is a main feature heavily advertised in their blog. Look by yourself:

https://nextcloud.com/blog/nextcloud-forms-to-keep-your-surveys-private/

It's not unreasonable to get pissed when 6 months after that blog post it doesn't support the latest release anymore.

Seems easier to blame Nextcloud

I have daily Borg backups held for at least one year but the problem is that the issue came out at least two weeks ago and nobody noticed. It's better to have nothing (customer gets error page when viewing useless survey that nobody is watching) rather to restore such a old backup (everyone loses 2-4 weeks of data)

You can't update it in the web app. You need to do a docker-compose pull followed by docker-compose up -d

Docker is kind of a giant mess in my experience. The trick to it is creating backup plans to recover your data when it fails.

Thats the trick for any production service. Especially when you do an update.

This is why you have a staging environment and backups for the very rare case where something goes wrong. I've only had to roll back a container maybe 3 times in 5 years at this point and can just roll back to a previous snapshot in a few minutes.

I wouldn't recommend Docker for a production environment either, but there are plenty of container-based solutions that use OCI compatible images just fine and they are very widely used in production. Having said that, plenty of people run docker images in a homelab setting and they work fine. I don't like running rootful containers under a system daemon, but calling it a giant mess doesn't seem fair in my experience.

That reminds me of work. I'm old, young me has been through the mistakes and the pain of wanting to control and self-host everything.

Now I manage a team of young idealists who have not yet been burned sufficiently hard by reality and I feel like I spend half of my time denying them permission to add new self-hosted services to our stack.

Just last month a young padawan was pissed at the spent on an external auth service and had been pushing hard for a self hosted OSS solution which he was convinced he could handle by himself (which was most likely true, from a purely technical standpoint).

Since he wouldn't let it go, I "punished" him by having him spend one day in excel and powerpoint to prepare a cost benefit analysis to present to the architecture review board, including server cost, backups, redundancy, security, monitoring, pen-testing, auditing, his time and all the bells and whistles we needed to be compliant with all the ISO-x we have to be. (we're in a banking related field).

Our estimated internal cost ended up about 6x the one of the SASS solutions and still wasn't as reliable.

Most people don't understand the amount of effort it requires to run a secure & reliable system and if I had a dollar for everytime I heard it's as simple as "docker run", I could retire early.

Docker is probably the simplest way to get a working deployment, since there's a lot of moving pieces in a Nextcloud install.

Though, it's not going to automatically update itself unless you've made a poor choice for a production environment configuration, which sounds like what happened here.

(Even using a latest tag isn't really a problem until/unless you re-pull the image to do the upgrade. And/or have configured something to automatically update your shit, but again, don't do that in production.)

Nextcloud is also annoying in that updating the base won't pull all the apps to a current version, so you have to know what's going to break before you update the base so you can then update the apps as needed. Which, again, can't just be left up to automatic updates.

Yes, docker is the best way. Anything else is hell. It is still painful with docker but at least it is manageable