Useful Linux commands
Tags: Linux
Last update: Aug 2022
- Compare binary files within the console and highlight those areas which differ. Vim and Colordiff have to be installed before.
diff -y <(xxd file.one) <(xxd file.two) | colordiff
2. Using a harddisk device for storing encrypted data with luks:
# creating luks partition sudo cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 -y /dev/sdX # mount luks partition sudo cryptsetup luksOpen /dev/sdX yourDeviceName # create a filesystem within this luks partition sudo mkfs.btrfs -L yourLabel /dev/mapper/yourDeviceName #(if you want btrfs) sudo mkfs.ext4 /dev/mapper/yourDeviceName #(if you want ext4) # if you want to mount it sudo mount /dev/mapper/yourDeviceName /yourMountPoint # generally speaking: mounting your filesystem sudo cryptsetup luksOpen /dev/sdX yourDeviceName sudo mount /dev/mapper/yourDeviceName /yourMountPoint # generally speaking: unmounting your filesystem sudo umount /dev/yourMountPoint sudo cryptsetup luksClose yourDeviceName
3. Mirroring with rsync from /source to /destination. Compression is great when backing up over the Internet.
rsync --compress --compress-level=9 --human-readable -av --progress --delete --log-file=/path/to/log.file /source /destination
4. Wipe a file with zeros:
shred -fvzun 0 file
5. Wipe a directory recursively with zeros:
srm -rvs directory
6. Backup of harddisks into compressed image file. Due to its multithreading capability pigz is much faster than gzip during compression.
sudo dd if=/dev/sdX status=progress bs=1M | pigz > harddisk.img.gz
7. Restore this prior zipped image file and write it directly to disk:
sudo gunzip -c image.gz | sudo dd of=/dev/sdX status=progress bs=1M
8. Recover a Linux Soft-RAID md/5:
sudo mdadm --assemble --scan sudo cryptsetup luksOpen /dev/md/5 temp sudo mount /dev/mapper/temp /your_path
9a. Mount a Linux Soft-RAID manually:
sudo mdadm --examine /dev/sdX sudo mdadm -A -R /dev/md9 /dev/sdX sudo mount /dev/md9 /openedRaidPartition
9b. And close it again:
sudo umount /openedRaidPartition sudo mdadm -S /dev/md9
10. List all loadable kernel modules:
find /lib/modules/$(uname -r) -type f -name \*.ko
11. Template for configuring a fresh manjaro xfce installation:
#update system sudo pacman -Syyu #install zshell sh -c "$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)" sudo pacman -S powerline-fonts #install virtualbox sudo pacman -S virtualbox #copy fonts which you need into /usr/share/fonts and run sudo fc-cache #fixing the Manjaro Linux XFCE text shadow problem on the desktop xfconf-query -c xfce4-desktop -p /desktop-icons/center-text -n -t bool -s false
12. Nice to have Manjaro packages:
gufw (firewall gui) chromium (browser) synapse (launcher) psensor (temperature) ether (pendrive tool) file-roller p7zip zip unzip unrar tilda (background terminal)
13. Laravel development quickstart:
git clone project.git cd project composer install composer update cp .env.example .env php artisan key:generate php artisan serve --port=8000
14. Restricting sFTP:
Add to /etc/ssh/sshd_config:
Subsystem sftp internal-sftp Match Group sftpusers ChrootDirectory %h ForceCommand internal-sftp AllowTcpForwarding no X11Forwarding no
Adding user and change configuration:
useradd newuser usermod -s /bin/false newuser chown root:root /home/newuser chmod 755 /home/newuser mkdir /home/newuser/writablefolder chown newuser:newuser /home/newuser/writeablefolder chmod 755 /home/newuser/writeablefolder groupadd sftpusers usermod -G sftpusers newuser service sshd restart service vsftpd restart
15. Replacing a broken harddisk (/dev/sda) with two partitions with RAID1 (md0 + md1):
#after having replaced the harddisk; copying MBR partition table sfdisk -d /dev/sdb | sfdisk /dev/sda #adding partitions to existing RAID-array md0 and md1: sudo mdadm /dev/md0 -a /dev/sda1 sudo mdadm /dev/md1 -a /dev/sda2 #reinstalling grub on both disks again, leave options empty, select two disks (/dev/sda + /dev/sdb) sudo dpkg-reconfigure grub-pc
16. Rsync command which doesn’t copy everything every time while preserving times:
rsync -avP source destination #a = archive mode #v = verbose #P = show progress
17. Bash: iterate through all files within a specific folder and count its files
for i in /folderpath /folderpath ; do
echo -n $i": " ;
(find "$i" -type f | wc -l) ;
done
18. copying KVM to PROXMOX
# converting image
qemu-img convert -f qcow2 -O raw image.qcow2 image.img
# creating VM in proxmox
qm create 120 --bootdisk scsi0
# importing disk to someStorage (as named in "pvesm status")
# importdisk adds the image as unused disk
qm importdisk 120 someImage.img someStorage
# attaching the image to the VM. still needs to be marked as boot device by hand
qm set 120 --scsi0 someStorage:vm-120-disk-019. repair a degraded ZFS pool/raid1 in PROXMOX
# "zpool status" output during normal operation
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-SanDisk_SDSSDH3_2T00_193732800276-part3 ONLINE 0 0 0
ata-SanDisk_SDSSDH32000G_192970802134-part3 ONLINE 0 0 0
errors: No known data errors# "zpool status" of degraded state if we unplug /dev/sdb manually
pool: rpool
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
ata-SanDisk_SDSSDH3_2T00_193732800276-part3 ONLINE 0 0 0
ata-SanDisk_SDSSDH32000G_192970802134-part3 UNAVAIL 3 212 0
errors: No known data errorsWe see that the disk with ID ata-SanDisk_SDSSDH32000G_192970802134-part3 is not available any more. We have to do the following steps:
- shutdown the server
- plug in a brand new replacement disk
- boot the server up again
You’ll see now another status:
pool: rpool
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
ata-SanDisk_SDSSDH3_2T00_193732800276-part3 ONLINE 0 0 0
14464946144281260868 UNAVAIL 0 0 0 was /dev/disk/by-id/ata-SanDisk_SDSSDH32000G_192970802134-part3We need to prepare the new drive and replace it:
# copy partition table from good /dev/sda to new /dev/sdb
sgdisk -R /dev/sdb /dev/sda
sgdisk -G /dev/sdb
# copy the bios and efi boot partitions
dd if=/dev/sda1 of=/dev/sdb1 bs=512
dd if=/dev/sda2 of=/dev/sdb2 bs=512
# replace command
zpool replace rpool ata-SanDisk_SDSSDH32000G_192970802134-part3 /dev/sdb3Keep in mind: Never use the detach command to remove a disk from a pool, because by adding it again you might be not careful enough to add it in mirror function. The default behaviour is to add a disk in striping mode (raid0)! You’ll have to backup everything and to reinstall a mirrored array again from scratch. Useful commands are:
# show raid status
zpool status
# show all your disk IDs
ls -l /dev/disk/by-id/20. How to fix the locale issue
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
sudo locale-gen en_US.UTF-8 sudo localedef -i en_US -f UTF-8 en_US.UTF-8 sudo nano /etc/environment +export LANGUAGE=en_US.UTF-8 +export LANG=en_US.UTF-8 +export LC_ALL=en_US.UTF-8
Leave a Reply
Want to join the discussion?Feel free to contribute!