User avatar
LilaHexe @LilaHexe@fedi.lilahexe.top
1d
What should I use to backup my crap?

Needs:
- linux support
- easily automated
- encryption
- incremental backups
- dedupe/compression
- low memory usage, even with lots of files (so no restic)
- rclone backend support

Wants:
- no slop
17
5
0
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe restic + resticprofile
1
0
0
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe Wait you said restic doesn't have low memory usage? I haven't experienced that
1
0
0
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe Oh and also what do you define as "low"
1
0
0
0
User avatar
LilaHexe @LilaHexe@fedi.lilahexe.top
1d
@star My server OOM'd last night because of restic, taking everything down with it. I've seen restic use like 3GB at times.

Also restic has slop in it...
2
0
0
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe If you use systemd to schedule backups, you could attempt 
[Service]
MemoryHigh=whateverM


And with resticprofile you are also able to set the global process priority of your backups to "low" or "background", which probably also improves things. I could also imagine that, if possible, pivoting to more frequent backups could help, if your services are creating a lot of files. AFAIK, a huge amount of files is the scenario where restic starts to eat memory for breakfast. But, since backups are incremental and all, hourly backups might not cost you more than, say, daily backups, and might reduce the amount of files backed up with each individual backup run
1
0
1
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe icydk, memory high doesn't OOM, it is just a soft ceiling after which the process gets slowed down and might get taken memory away from it. I haven't checked out how restic deals with this though, but might be worth trying!
1
0
1
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe I thought about this some moreeeeee and maybe this could also help :3
[Service]
MemoryHigh=750M
MemoryMax=1500M
MemorySwapMax=infinity



But I am unsure! I haven't tested this out myself and my cgroup knowledge is limited at best
1
0
1
0
User avatar
LilaHexe @LilaHexe@fedi.lilahexe.top
1d
@star Thanks for the tips 💜

I'm using backrest rn, but might switch to systemd
0
0
1
0
User avatar
Alba 🌸 v_pat @mildsunrise@tech.lgbt
1d
@LilaHexe @star I'm curious, how many files are we talking about here? just the order of magnitude

re. the slop, I am now one unit sadder after knowing this information
1
0
1
0
User avatar
LilaHexe @LilaHexe@fedi.lilahexe.top
1d
@mildsunrise @star About 660,000 files
2
0
1
0
User avatar
Celeste_transbian flori_ava_star:~cursor_blinkingbadge-petting-encouraged badge-biting-allowed @star@amazonawaws.com
1d
@LilaHexe @mildsunrise consider adding 10,000 more? neocat_angel_pleading
💜1
0
2
2
1
User avatar
Alba 🌸 v_pat @mildsunrise@tech.lgbt
13h
@LilaHexe @star hmm weird, i have 2.2 million and it stayed at ~700MB of ram... I have --read-concurrency=8 but if anything I would expect that to increase usage, not decrease it
0
0
1
0