Tags: Linux
Last update: Aug 2022
- Compare binary files within the console and highlight those areas which differ. Vim and Colordiff have to be installed before.
diff -y <(xxd file.one) <(xxd file.two) | colordiff
2. Using a harddisk device for storing encrypted data with luks:
# creating luks partition
sudo cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 -y /dev/sdX
# mount luks partition
sudo cryptsetup luksOpen /dev/sdX yourDeviceName
# create a filesystem within this luks partition
sudo mkfs.btrfs -L yourLabel /dev/mapper/yourDeviceName #(if you want btrfs)
sudo mkfs.ext4 /dev/mapper/yourDeviceName #(if you want ext4)
# if you want to mount it
sudo mount /dev/mapper/yourDeviceName /yourMountPoint
# generally speaking: mounting your filesystem
sudo cryptsetup luksOpen /dev/sdX yourDeviceName
sudo mount /dev/mapper/yourDeviceName /yourMountPoint
# generally speaking: unmounting your filesystem
sudo umount /dev/yourMountPoint
sudo cryptsetup luksClose yourDeviceName
3. Mirroring with rsync from /source to /destination. Compression is great when backing up over the Internet.
rsync --compress --compress-level=9 --human-readable -av --progress --delete --log-file=/path/to/log.file /source /destination
4. Wipe a file with zeros:
shred -fvzun 0 file
5. Wipe a directory recursively with zeros:
srm -rvs directory
6. Backup of harddisks into compressed image file. Due to its multithreading capability pigz is much faster than gzip during compression.
sudo dd if=/dev/sdX status=progress bs=1M | pigz > harddisk.img.gz
7. Restore this prior zipped image file and write it directly to disk:
sudo gunzip -c image.gz | sudo dd of=/dev/sdX status=progress bs=1M
8. Recover a Linux Soft-RAID md/5:
sudo mdadm --assemble --scan
sudo cryptsetup luksOpen /dev/md/5 temp
sudo mount /dev/mapper/temp /your_path
9a. Mount a Linux Soft-RAID manually:
sudo mdadm --examine /dev/sdX
sudo mdadm -A -R /dev/md9 /dev/sdX
sudo mount /dev/md9 /openedRaidPartition
9b. And close it again:
sudo umount /openedRaidPartition
sudo mdadm -S /dev/md9
10. List all loadable kernel modules:
find /lib/modules/$(uname -r) -type f -name \*.ko
11. Template for configuring a fresh manjaro xfce installation:
#update system
sudo pacman -Syyu
#install zshell
sh -c "$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"
sudo pacman -S powerline-fonts
#install virtualbox
sudo pacman -S virtualbox
#copy fonts which you need into /usr/share/fonts and run
sudo fc-cache
#fixing the Manjaro Linux XFCE text shadow problem on the desktop
xfconf-query -c xfce4-desktop -p /desktop-icons/center-text -n -t bool -s false
12. Nice to have Manjaro packages:
gufw (firewall gui)
chromium (browser)
synapse (launcher)
psensor (temperature)
ether (pendrive tool)
file-roller
p7zip zip unzip unrar
tilda (background terminal)
13. Laravel development quickstart:
git clone project.git
cd project
composer install
composer update
cp .env.example .env
php artisan key:generate
php artisan serve --port=8000
14. Restricting sFTP:
Add to /etc/ssh/sshd_config:
Subsystem sftp internal-sftp
Match Group sftpusers
ChrootDirectory %h
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no
Adding user and change configuration:
useradd newuser
usermod -s /bin/false newuser
chown root:root /home/newuser
chmod 755 /home/newuser
mkdir /home/newuser/writablefolder
chown newuser:newuser /home/newuser/writeablefolder
chmod 755 /home/newuser/writeablefolder
groupadd sftpusers
usermod -G sftpusers newuser
service sshd restart
service vsftpd restart
15. Replacing a broken harddisk (/dev/sda) with two partitions with RAID1 (md0 + md1):
#after having replaced the harddisk; copying MBR partition table
sfdisk -d /dev/sdb | sfdisk /dev/sda
#adding partitions to existing RAID-array md0 and md1:
sudo mdadm /dev/md0 -a /dev/sda1
sudo mdadm /dev/md1 -a /dev/sda2
#reinstalling grub on both disks again, leave options empty, select two disks (/dev/sda + /dev/sdb)
sudo dpkg-reconfigure grub-pc
16. Rsync command which doesn’t copy everything every time while preserving times:
rsync -avP source destination
#a = archive mode
#v = verbose
#P = show progress
17. Bash: iterate through all files within a specific folder and count its files
for i in /folderpath /folderpath ; do
echo -n $i": " ;
(find "$i" -type f | wc -l) ;
done
18. copying KVM to PROXMOX
# converting image
qemu-img convert -f qcow2 -O raw image.qcow2 image.img
# creating VM in proxmox
qm create 120 --bootdisk scsi0
# importing disk to someStorage (as named in "pvesm status")
# importdisk adds the image as unused disk
qm importdisk 120 someImage.img someStorage
# attaching the image to the VM. still needs to be marked as boot device by hand
qm set 120 --scsi0 someStorage:vm-120-disk-0
19. repair a degraded ZFS pool/raid1 in PROXMOX
# "zpool status" output during normal operation
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-SanDisk_SDSSDH3_2T00_193732800276-part3 ONLINE 0 0 0
ata-SanDisk_SDSSDH32000G_192970802134-part3 ONLINE 0 0 0
errors: No known data errors
# "zpool status" of degraded state if we unplug /dev/sdb manually
pool: rpool
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
ata-SanDisk_SDSSDH3_2T00_193732800276-part3 ONLINE 0 0 0
ata-SanDisk_SDSSDH32000G_192970802134-part3 UNAVAIL 3 212 0
errors: No known data errors
We see that the disk with ID ata-SanDisk_SDSSDH32000G_192970802134-part3 is not available any more. We have to do the following steps:
- shutdown the server
- plug in a brand new replacement disk
- boot the server up again
You’ll see now another status:
pool: rpool
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
ata-SanDisk_SDSSDH3_2T00_193732800276-part3 ONLINE 0 0 0
14464946144281260868 UNAVAIL 0 0 0 was /dev/disk/by-id/ata-SanDisk_SDSSDH32000G_192970802134-part3
We need to prepare the new drive and replace it:
# copy partition table from good /dev/sda to new /dev/sdb
sgdisk -R /dev/sdb /dev/sda
sgdisk -G /dev/sdb
# copy the bios and efi boot partitions
dd if=/dev/sda1 of=/dev/sdb1 bs=512
dd if=/dev/sda2 of=/dev/sdb2 bs=512
# replace command
zpool replace rpool ata-SanDisk_SDSSDH32000G_192970802134-part3 /dev/sdb3
Keep in mind: Never use the detach command to remove a disk from a pool, because by adding it again you might be not careful enough to add it in mirror function. The default behaviour is to add a disk in striping mode (raid0)! You’ll have to backup everything and to reinstall a mirrored array again from scratch. Useful commands are:
# show raid status
zpool status
# show all your disk IDs
ls -l /dev/disk/by-id/
20. How to fix the locale issue
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
sudo locale-gen en_US.UTF-8
sudo localedef -i en_US -f UTF-8 en_US.UTF-8
sudo nano /etc/environment
+export LANGUAGE=en_US.UTF-8
+export LANG=en_US.UTF-8
+export LC_ALL=en_US.UTF-8
