Drive/sunetdrive
7
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
599 commits 5 branches 415 tags 1.9 MiB
Commit graph

280 commits

Author SHA1 Message Date
Micke Nordin 717350780f Typo 2024-06-05 11:17:12 +02:00
Micke Nordin 450f31e2d5 Remove drive resolvers 2024-05-21 19:46:20 +02:00
Magnus Andersson aa2a869203
Move gss prod to redis cluster 2024-05-21 17:14:03 +02:00
Micke Nordin 05814cfd24 proxysql network does not exist 2024-05-20 13:10:20 +02:00
Micke Nordin 472977fde4 Update config for lookup server 2024-05-20 10:44:37 +02:00
Magnus Andersson a16c9d68e8
Disable MultipartCopy with user bucket and restore scripts and primary bucket. 2024-05-08 13:31:22 +02:00
Micke Nordin 47bf118116 Merge branch 'main' into mandersson-nmhost 2024-05-07 13:09:29 +02:00
Micke Nordin 51a695ed9e Don't fail on problems with downtime for db either 2024-05-07 10:53:44 +02:00
Micke Nordin cb16f03710 Don't fail on problems with downtime 2024-05-07 10:50:55 +02:00
Magnus Andersson c774d67745
App and proxysql: Fixes for network_type: host 2024-04-24 21:32:25 +02:00
Magnus Andersson d5c6b9a5fd
proxysql: Do not create docker network with network_mode: host 2024-04-24 20:52:26 +02:00
Magnus Andersson 8d2c481569
Frontend node: Use netork mode host with dockerhost2 2024-04-24 20:44:41 +02:00
Micke Nordin b5b3885c30 Allways make config editable for occ 2024-04-22 12:05:07 +02:00
Micke Nordin e72c42924a Disallow external files to mount local storage 2024-04-22 11:08:41 +02:00
Magnus Andersson d07ed76d36
config.php: Increase davstorage request timeout to support bigger files. 2024-04-17 16:38:09 +02:00
Micke Nordin b4ce0046de Add jupyter site 2024-03-18 15:17:30 +01:00
Magnus Andersson 65abea7daa
Fix proxysql container name. 2024-03-18 11:08:54 +01:00
Magnus Andersson f1cac6ce35
Hardcode nextcloud container name. 2024-03-18 10:30:10 +01:00
Magnus Andersson cc6f2c8d7b
Maria Backup: Add fixed container name to mitigate name statndard change docker-compose 2/3 2024-03-18 08:41:11 +01:00
Magnus Andersson 4f6786363b
Hardcode name of mariadb container to make name consistent between docker compose 2 and 3. 2024-03-08 10:32:02 +01:00
Micke Nordin 1a70154efc Sites moved to i18n 2024-03-01 10:12:22 +01:00
Micke Nordin cbb6ce3aa3 Fix typo 2024-02-27 16:35:49 +01:00
Micke Nordin f511383a7a Add portal 2024-02-27 11:59:32 +01:00
Magnus Andersson db6bfcc517
Add focemfa as forcemfa group to template. 2023-12-17 23:18:02 +01:00
Micke Nordin 7dc9667bec Fix typo 2023-11-22 12:12:57 +01:00
Micke Nordin 6265472396 Make it possible to delete announcement with just subject 2023-11-22 11:52:58 +01:00
Micke Nordin 300e877b3a Make curl silent 2023-11-22 11:01:52 +01:00
Fredrik Kjellman 436ea4f1a5 update template for readability. SD-285 2023-11-10 12:40:00 +01:00
Micke Nordin a08f473954 remove subscript of type 2023-11-07 14:01:12 +01:00
Micke Nordin 9d5e53c0cc Merge branch 'stable' 2023-11-01 16:55:10 +01:00
Micke Nordin 9e4879d9c8 python3.9 -> python3 2023-11-01 15:20:23 +01:00
Fredrik Kjellman 3491a1a5f7 change MDU readme document. SD-285. 2023-10-31 13:49:37 +01:00
Richard Freitag 76bc7edaed Remove @ from mappingfile in prod and test and update regex to comply with Nextcloud keys 2023-10-24 14:46:37 +01:00
Richard Freitag 7d25ae0859 Remove @ from mappingfile in prod and update regex to comply with Nextcloud keys 2023-10-24 14:43:45 +01:00
Magnus Andersson 3cdf7d4084
Revert "4 backslashes in mappingfile-test.json.erb. Does not work." 
This reverts commit 14b6535efd.
 2023-10-23 15:13:11 +02:00
Magnus Andersson 14b6535efd
Test of adding 4 backslashes in mappingfile-test.json.erb to deal with json handling of backslashes. 2023-10-23 14:49:46 +02:00
Micke Nordin 0148a727ca Revert mapping file for prod 2023-10-23 12:36:10 +02:00
Micke Nordin e06a29a8a1 Revert mapping file for prod 2023-10-23 12:35:24 +02:00
Micke Nordin 5c5ed31f13 Remove test 2023-10-19 14:50:06 +02:00
Micke Nordin 21d9a5e3d0 Remove test 2023-10-19 14:49:46 +02:00
Micke Nordin 03fb0a23ac Nextcloud: Implement logs to stdout 2023-10-18 08:43:46 +02:00
Magnus Andersson 9e6321e743 backup-all-buckets: Add comment about safeguard 2023-10-18 08:43:46 +02:00
Magnus Andersson b1ea687f2d backup-all-buckets: Safeguard backup runs if a backup process on that bucket is already in progress. 2023-10-18 08:43:46 +02:00
Richard Freitag 33d49b7ec0 Change regex to send @.shh.se cases to external node 2023-10-18 08:43:46 +02:00
Richard Freitag f988081b39 Improved mappingfile that should work for all cases now 2023-10-18 08:43:22 +02:00
Micke Nordin ec9115df06 Nextcloud: Implement logs to stdout 2023-10-17 17:01:50 +02:00
Richard Freitag e39b6cbaed
Update mappingfile also in production 2023-10-13 13:10:49 +02:00
Richard Freitag 11607f208c
Change regex to send @.shh.se cases to external node 2023-10-13 13:10:16 +02:00
Richard Freitag 823e172278
Improved mappingfile that should work for all cases now 2023-10-13 13:09:39 +02:00
Magnus Andersson 6afbcf5beb
listusers.sh: disabel tls verify for rclone mkdir because we use ip address with webdav. 2023-10-13 11:05:02 +02:00
Magnus Andersson 059bd041e6
backup-all-buckets: Add comment about safeguard 2023-10-12 09:21:57 +02:00
Magnus Andersson 5df8dba25c
backup-all-buckets: Safeguard backup runs if a backup process on that bucket is already in progress. 2023-10-12 09:03:48 +02:00
Richard Freitag 5472b767c8 Update mappingfile also in production 2023-10-10 12:30:53 +02:00
Richard Freitag e990b05401 Change regex to send @.shh.se cases to external node 2023-10-10 12:30:53 +02:00
Magnus Andersson ca27f46ffe
listusers.sh: disabel tls verify for rclone mkdir because we use ip address with webdav. 2023-10-10 10:48:43 +02:00
Richard Freitag 829f3f01c7 Improved mappingfile that should work for all cases now 2023-10-05 12:10:21 +01:00
Micke Nordin 37863c7a17
Use rclone for reva 2023-09-28 14:48:30 +02:00
Micke Nordin 3bac4ac9f3
Fix 2023-09-26 14:38:44 +02:00
Micke Nordin 216c706457
Fix parameters 2023-09-26 14:25:10 +02:00
Micke Nordin f606d6c1b6
Update config 2023-09-26 14:15:59 +02:00
Micke Nordin 3d4025e738
Set seccomp:unconfined 2023-09-26 09:44:56 +02:00
Micke Nordin b4b90799e6
Update reva version 2023-09-26 09:24:36 +02:00
Micke Nordin d2ced8d1cd
Backups: This script does not seem used and is confusing 2023-09-25 09:34:19 +02:00
Magnus Andersson b9e35dcf92 Give multinode a sixmonts backup policy. 2023-09-20 10:26:01 +02:00
Micke Nordin b5f8abf934
Allow source missmatch after changing bucket name 2023-09-12 16:50:48 +02:00
Micke Nordin 6d33b53a92
Fix backup directory for multinode script as well 2023-09-12 16:37:36 +02:00
Micke Nordin 631f3616c6
Create bespoe directory, and clean up afterwards 2023-09-12 16:35:52 +02:00
Micke Nordin 174c72eb70
BACKUPDB: Fix bug that put backup in a file instead of a directory 
We found a bug that moved the backups to a file, but we want that to be
a directory, meaning that all the files were overwritten one by one, and
we were not taking the backups as we though.
 2023-09-12 16:19:17 +02:00
Micke Nordin 22f5dfac62
Allways backup before restarting 2023-09-02 12:50:08 +02:00
Micke Nordin 7207ef2bbf
Allways backup the db 2023-09-02 12:30:16 +02:00
Micke Nordin c7bd0fa965
Try again 2023-09-02 12:19:33 +02:00
Micke Nordin d540c90afa
Try again 2023-09-02 12:15:54 +02:00
Micke Nordin b927716aad
Fix comment 2023-09-02 12:12:59 +02:00
Micke Nordin ac6430df51
Add announce script to script servers 2023-09-02 12:08:17 +02:00
Micke Nordin 88b3c1b4fb
Merge branch 'main' into testing 2023-09-02 09:29:06 +02:00
Micke Nordin 5bc0ddfa00
Add admin password as env variable to image 2023-09-02 09:28:39 +02:00
Micke Nordin f56a5c6eae Add get drive customers script to all servers 2023-08-16 12:53:08 +02:00
Micke Nordin 598be8c2f1
Add full support for common infra 2023-07-20 11:23:48 +02:00
Micke Nordin 54de0adb0f
Remove replicate readwrite db 2023-07-18 15:52:33 +02:00
Micke Nordin 1152fd972f
Add 'gss.username_format' => 'sanitize', 2023-07-06 14:50:47 +02:00
Micke Nordin befc3207d9
Fix conficting bashrc declarations 2023-06-15 19:24:31 +02:00
Micke Nordin 1e23d134f1
Add fqdn to PS1 2023-06-13 14:15:53 +02:00
Micke Nordin e0139b0765
BAckups: Fix db backups 
DB backup dir clashed with statistics script storage location, and also
some permission errors.

diff --git a/templates/script/backupdb.erb.sh b/templates/script/backupdb.erb.sh
index b35acbc..1eb7992 100644
--- a/templates/script/backupdb.erb.sh
+++ b/templates/script/backupdb.erb.sh
@@ -6,11 +6,16 @@ sleep $((16#$(ip a | grep "link/ether" | head -1 | awk -F ':' '{print $6}' | awk
 number_of_full_to_keep="<%= @full_backup_retention %>"

 backup="${1}"
+if [[ -z ${backup} ]]; then
+  backup="backup1.$(hostname -d)"
+fi
 if ! [[ ${backup} =~ backup1.*sunet.se$ ]]; then
 	echo "Usage: ${0} <fqdn of backup server>"
 	echo "Example: ${0} backup1.sunet.drive.sunet.se"
 fi
 backup_dir="/opt/backups"
+backup_dir_long="${backup_dir}/backups"
+mkdir -p ${backup_dir}
 bucket="db-backups"
 mirror="<%= @customer %>-<%= @environment %>-mirror"
 if [[ ${mirror} =~ common-(test|prod)-mirror ]]; then
@@ -23,11 +28,8 @@ ssh ${backup} "sudo /home/script/bin/backup_db.sh"
 echo "Cleaning up old backups for ${backup}"
 ssh ${backup} "sudo /home/script/bin/purge_backups.sh /opt/mariadb_backup/backups/"
 echo "Copying backups here"
-mkdir -p ${backup_dir}
-scp script@${backup}:/opt/mariadb_backup/backups/$(date +%Y/%m/%d)/*.gz ${backup_dir}
+scp script@${backup}:/opt/mariadb_backup/backups/$(date +%Y/%m/%d)/*.gz ${backup_dir_long}
 echo "Copying backups to remote bucket"
 rclone mkdir ${mirror}:${bucket}
 duplicity --full-if-older-than 1M --tempdir /mnt --archive-dir /mnt --no-encryption ${backup_dir} rclone://${mirror}:/${bucket}
 duplicity remove-all-but-n-full ${number_of_full_to_keep} --tempdir /mnt --archive-dir /mnt --force rclone://${mirror}:/${bucket}
-echo "cleaning up"
-rm -r ${backup_dir}
diff --git a/templates/scriptreceiver/backup_db.erb.sh b/templates/scriptreceiver/backup_db.erb.sh
index 025a338..3e15f25 100644
--- a/templates/scriptreceiver/backup_db.erb.sh
+++ b/templates/scriptreceiver/backup_db.erb.sh
@@ -11,5 +11,7 @@ else
 fi

 docker exec ${container} /do_backup.sh ${customer}
+chown root:script /opt/mariadb_backup/
+chmod 750 /opt/mariadb_backup/
 chmod 755 ${backupdir}
 chown -R script:root ${backupdir}
 2023-06-13 12:15:19 +02:00
Micke Nordin 0f8f27773b
Add support for replicating from multinode dbs 2023-06-12 09:31:41 +02:00
Micke Nordin e7ffd6fac3
Add nordu.net 2023-06-08 16:43:05 +02:00
Micke Nordin 3531be828e
Add nordu.net 2023-06-08 16:32:43 +02:00
Micke Nordin 24b0b1eedc
Revert "Use nce image" 
This reverts commit 20471ebaa8.
 2023-06-01 11:47:16 +02:00
Micke Nordin 20471ebaa8
Use nce image 2023-06-01 11:44:50 +02:00
Fredrik Kjellman 3dab127050 Merge branch 'main' into stable 2023-05-08 09:15:12 +02:00
Fredrik Kjellman fc4b51b4ac update mapping file to fix problems for shh.se 2023-05-08 09:14:00 +02:00
Fredrik Kjellman a14a71f5d6 Merge branch 'main' into stable 2023-05-03 15:32:38 +02:00
Fredrik Kjellman 52c11917ce modify mappingfile for GSS to help SHH to login. 2023-05-03 15:00:24 +02:00
Micke Nordin ca2cf7e3f2
Allow file locks to be debug logged for sunet test 2023-04-27 16:44:06 +02:00
Micke Nordin 75f75732e7
Fix repair command 2023-04-26 11:22:25 +02:00
Micke Nordin 8ca66c10ad
Add more repair commands and use tty for occ 2023-04-26 09:26:46 +02:00
Micke Nordin 7d10d168bd
Fix nc_upgrade_command 2023-04-25 17:48:13 +02:00
Micke Nordin 036dcf010c
Add gss 2023-04-24 16:39:09 +02:00
Micke Nordin 3b5d14a189
Try again 2023-04-24 16:34:37 +02:00
Micke Nordin e840903eae
Can I force string 2023-04-24 16:30:12 +02:00
Micke Nordin db516e3b72
New name 2023-04-24 16:27:21 +02:00
Micke Nordin a9695aeee1
Two passes 2023-04-24 16:25:37 +02:00
Micke Nordin 713f12e1d4
Terminate regex 2023-04-24 16:23:01 +02:00
Micke Nordin 568cb87ad6
Trim version 2023-04-24 16:21:22 +02:00
Micke Nordin 709d2165ac
Gsub 2023-04-24 16:13:11 +02:00
Micke Nordin 37e419cf75
Try regsubst 2023-04-24 16:06:55 +02:00
Micke Nordin 54868e8518
Yet another try 2023-04-24 15:59:47 +02:00
Micke Nordin 20d1558866
Yet another try 2023-04-24 15:56:22 +02:00
Micke Nordin f1276d52d3
Fingers crossed 2023-04-24 15:51:49 +02:00
Micke Nordin 9b44a99135
Print debug to file 2023-04-24 15:47:56 +02:00
Micke Nordin 9ec76cfc7f
Yet another try 2023-04-24 15:39:32 +02:00
Micke Nordin 71fd8a3287
Yet another try 2023-04-24 15:34:02 +02:00
Micke Nordin f95675f6ee
Yet another try 2023-04-24 15:27:51 +02:00
Micke Nordin e06b7ffb1c
Try to lookup fact yet another way 2023-04-24 15:23:57 +02:00
Micke Nordin 1045898d40
Diffent fact syntax 2023-04-24 15:13:10 +02:00
Micke Nordin 4af4a23d1b
Use new facts in template 2023-04-24 15:06:51 +02:00
Micke Nordin c5e5279f0e
Add reference to self 2023-04-24 12:10:30 +02:00
Micke Nordin 165415b725
Less dangerous pruning script 2023-04-24 10:47:36 +02:00
Micke Nordin f2f919a59c
Lower further 2023-04-05 14:20:52 +02:00
Micke Nordin 801c5a2e85
Lower interned string value 2023-04-05 14:13:00 +02:00
Micke Nordin 4948ada609
Use facter 2023-04-05 12:26:11 +02:00
Micke Nordin e220a1b1c6
Fix htaccess complaint 2023-04-05 08:55:41 +02:00
Micke Nordin bf5e5adf10
Apache would not start with 128mb 2023-04-05 08:27:53 +02:00
Micke Nordin 0b267c07c5
Fix warning about interned strings on some nodes 2023-04-05 08:08:31 +02:00
Micke Nordin e3dbe10187
Sort config and add entries to disable spurious warnings 2023-04-04 14:29:20 +02:00
Micke Nordin d93cce4fc0
Handle mau 2023-04-03 14:11:28 +02:00
Micke Nordin 6dfe1496a4
Purge binlogs regularly 2023-03-24 16:26:56 +01:00
Micke Nordin 8386df91de
Fix for multinode 2023-03-24 14:36:07 +01:00
Micke Nordin 6de5312e7c
Support editable config 2023-03-24 13:24:23 +01:00
Micke Nordin af020c77aa
New image 2023-03-24 11:47:55 +01:00
Micke Nordin cdfd3b9126
Clear scriptherder script 2023-03-22 08:28:28 +01:00
Micke Nordin 4be1448dab
Remove swap 2023-03-22 08:01:18 +01:00
Micke Nordin 38807e62a5
Try to configure all customer 2023-03-21 18:26:23 +01:00
Micke Nordin 238172397b
Correct network settings 2023-03-21 17:26:36 +01:00
Micke Nordin 3c1ee2ba1d
Fix networks 2023-03-20 09:40:19 +01:00
Micke Nordin 4963afd5f6
Handle clean up in a different way 2023-03-13 16:33:51 +01:00
Micke Nordin 18578817f1
Correct dir for backups 2023-03-13 16:27:36 +01:00
Micke Nordin 3fb6164501
Typo 2023-03-13 16:23:51 +01:00
Micke Nordin 392e83188b
Fix backups for new multionode dbs 2023-03-13 16:20:03 +01:00
Micke Nordin 810c095bda
Add support for maintenance:mode with readonly config 2023-03-13 14:09:43 +01:00
Micke Nordin 09d3f72162
Add support for maintenance:mode with readonly config 2023-03-13 13:59:53 +01:00
Micke Nordin d94ecc9a26
Fix up container name 2023-03-13 13:26:05 +01:00
Micke Nordin 284bc8ad8e
Rename default container 2023-03-13 13:23:19 +01:00
Micke Nordin abd56622e7
Fix typo 2023-03-13 12:49:03 +01:00
Micke Nordin e9e70c5b29
Add in support for fullnode customers to folders in userbuckets 2023-03-13 12:41:23 +01:00
Micke Nordin d09dc50d0f
Enable multinode servers to use external db 2023-03-10 12:50:55 +01:00
Micke Nordin 25d8d94b70
Fix container name 2023-03-10 10:19:05 +01:00
Micke Nordin 10c44fe5fc
Fix container name 2023-03-10 10:05:21 +01:00
Micke Nordin 00256a7259
Use python 3.9 2023-02-28 14:24:11 +01:00
Micke Nordin 23af9a6a24
Network mode host not compatible with ports 2023-02-28 14:17:49 +01:00
Micke Nordin 45762b076a
Add proxysql network 2023-02-28 14:06:17 +01:00