Etcher: AMAZING Unetbootin Alternative for Ubuntu/Debian/Linux/Windows

Unetbootin is FURIOUSLY difficult to install on Ubuntu, Debian and even Windows.

I tried the ~5MB unetbootin-linux-xxx.bin file which FAILED.

I tried the ~23MB unetbootin-linux.tar.gz source code which FAILED.

I tried WinUSB which failed.

I tried to get USB-Image Writer, which also failed.

Finally found Etcher, fully open source bootable SD & bootable USB image writer. Even better than standard dd method which lacks progress information.

Grab it here on Windows, Linux, even Mac: https://etcher.io/

Etcher unetbootin alternative

EASY: Move OS & Files from HDD to SSD – Debian, Ubuntu, Fedora, Arch (SMALL SSD)

This tutorial is people who will ADD an SSD to their system.
This NOT for people who will REPLACE their HDD completely with their SSD.

This is by far the easiest, most foolproof, and safest way to transfer 1 linux installation (Debian, Ubuntu, Fedora, Arch, Kali etc.) to a brand new SSD.

For the transfer commands, skip to 3. Transfer Files Section.

Backstory

My HDD is 1TB and my new SSD is a 120GB Samsung 850 EVO.

I bought a Samsung 850 EVO for my laptop and I also purchased an SSD Caddy. The SSD caddy was supposedly “universal” but still 2m or so off and did not fit into the CD-ROM tray. I dismantled the caddy and used one half the caddy to secure the drive in place.

I read that my laptop had a SATA III port where the normal HDD drive belongs, but the CD-ROM port was only a SATA II connection. Therefore, I replaced the old HDD with the SSD and moved the HDD into the caddy.

Samsung 850 PRO vs EVO was another small consideration, and the real difference seems only to be the extended warranty (5 year vs 10 year)… In 10 years from now I can only imagine what Samsung will have invented…

1. SSD Security: File Deletions & Securely Erasing SSDs.

As as is true with all drive wiping/erasing: wiping the entire drive (several times over) is the most offensive way to overwrite storage on a drive. SSDs operate differently to HDDs and whether or not files are deleted, even after filling them with /dev/random and /dev/zero, so the easiest way to prevent important files from being somewhat “recoverable” is to keep them on the HDD.

2. What size SSD should I get?

Not everything from my 1000GB HDD will fit on my new 120GB SSD, nor do I want this to happen because I have two drives now and the cost of having everything on my SSD is much higher because SSDs are pretty expensive. I also don’t want important documents on the SSD because I want to know that I can securely erase them at any time.

  • If you’re replacing your hard disk drive, you’ll need a big SSD.
  • If you’re adding to your hard disk drive (having 2x drives) you don’t really need a big SSD.

I chose the Samsung EVO 120GB SSD because this is more than enough to run Debian/Ubuntu/Arch. Moreover, it will force me to keep my important documents “secure” on the HDD, whilst still achieving a surreal ~550mb/s read & write speed from the new SSD.

Moving files onto the new SSD:

– Operating System (/boot folder, specifically initrd.img & vmlinuz)

– /bin, /lib, /usr, /var etc.

– Applications (FireFox, Chromium, Photoshop, Illustrator, PlayOnLinux, Libre Calc, Virtual Box & Images)

Keeping files on the old HDD:

– Swap memory partition

– Most of my documents

– Most of my images

– Encrypted folders

SSD data recovery is an important consideration as well. Writing and deleting files on an SSD is unlike an HDD. My mission critical documents will stay on my HDD. Since installing my SSD & HDD I have successfully set it up so that:

– I use applications from the SSD

– Open files stay in RAM

– Files Save & Load from the HDD

3. Moving Debian or Ubuntu to SSD [VERY SIMPLE]

rsync is very fast method of file transferring that will keep preserve:

  • folder structure
  • folder & file ownership
  • folder & file permissions

It will also “pick up where you left off” so that you can run it in several times until its finished.

We will use 2 options while using rsync:

-a which means archive mode.

-P which means –partial (continue where left off) and –progress (show progress)

Archive mode is equivalent to all of -rlptgoD which makes sure it’s a mirror if exactly what it’s copying.

Step 1: Mount your SSD

Plug it in, if it doesn’t show up, you may need to format it. Open GParted or Gnome Disks (You’ll have one of them) and format the drive. Gnome Disks will show the drive on the left panel: select it, click the “gear” button at the top, format as ext4. GParted will have a drop down menu in the top right: select your drive (identify by GB size), right click on the colored box and format to ext4.

Step 2: Open the SSD folder

Get into the SSD and note path that the files are located (should be empty, might have lost&found). On my system the SSD is at /mnt/SAMSUNG/. It might be /media/SSD/ or something like that. Open a terminal and execute df to find where its Mounted on.

Copy the path and paste it at the end of one of the below rsync commands that you will use.

Step 3: Rsync the OS, folder & files to the new drive

If you have a really big SSD and you want to copy everything from your old HDD to your SSD use this command (make sure your SSD is going to big enough!)

1
sudo rsync --exclude="mnt" --exclude="lost+found" --exclude="sys" --exclude="proc" --exclude="cdrom" --exclude="media" -aP / /mnt/SAMSUNG/

If you have a small SSD (this one ignores home temporarily).

1
sudo rsync --exclude="home" --exclude="mnt" --exclude="lost+found" --exclude="sys" --exclude="proc" --exclude="cdrom" --exclude="media" -aP / /mnt/SAMSUNG/

In the command above, we excluded “home” for the moment, because there’s some big files in there. When the above has finished, choose what you’d like to keep on the HDD. I kept Pictures, Downloads, Documents, PlayOnLinux Drives, Wine data Virtual Box Drives on the HDD. Replace “user” below with your username.

1
sudo rsync --exclude="Pictures" --exclude="Downloads" --exclude="Documents" --exclude=".PlayOnLinux" --exclude=".wine" --exclude="Virtual*" -aP /home/user/ /mnt/SAMSUNG/home/user

After your Operating System has been moved the SSD, we need to create a few more folders.

1
cd /
1
2
3
4
5
sudo mount -o bind /dev /mnt/SAMSUNG/dev
sudo mount -o bind /sys /mnt/SAMSUNG/sys
sudo mount -t proc /proc /mnt/SAMSUNG/proc
sudo cp /proc/mounts /mnt/SAMSUNG/etc/mtab
sudo chroot /media/user/SAMSUNG/ /bin/bash

If you are logged in as “root” in the new drive, it is good to go.

1
exit

Step 4: Reinstall grub

 

SUPER SIMPLE: Migrate/Move VestaCP Installation (10 seconds or less!)

I’ve moved VestaCP installations so fast that by the time a customer started the checkout page in Sydney, they submitted their order to Amsterdam.

This is best achieved with Cloudflare as there will be absolutely no downtime, whatsoever.

First make sure there is a new VestaCP installation on your new server. If you have a busy website, having the  new server ready to go will mean you will have less than 10 seconds of downtime.

Alternatively, you may be adding an old VestaCP installation to another VestaCP installation (maybe you realised how powerful & resource un-intensive VestaCP is, especially without exim + dovecot + spamassassin + clamav)

Will you be using admin as the account on the new server?

Create a fresh backup on the OLD SERVER

SSH into the server you want to close down and run

1
v-backup-user admin

When it’s done, you’ll see the backup timestamp and file size.

v-backup-user-admin

Note the DATE of the backup above.

Open a new terminal and SSH into the NEW SERVER that you want to move VestaCP to.

Run following command

1
scp -oStrictHostKeyChecking=no [email protected]:/home/backup/admin.2016-10-30.tar /home/backup/

Input the password of the old server and it will securely copy the Old Vest to the New Vesta backup folder.

Once it’s downloaded, restore the user called admin

1
v-restore-user admin admin.2016-10-30.tar

Your New Server will have been migrated!

SUPER FAST VESTACP MIGRATION METHOD – MILLISECOND DOWNTIME

Open Cloudflare account that holds the website you are migrating.
Open two terminals
In Terminal 1 SSH into your Old Server.
In Terminal 2 SSH into your New Server.
Make sure VestaCP is installed on the New Server (Terminal 2).
In the Terminal 1 Old Server execute

1
date

Note down the date, for example, “Sun Oct 30 00:22:33 UTC 2016”.
Now in the same Terminal 1 Old Server type (but don’t press enter yet!)

1
v-backup-user admin

In Terminal 2 New Server type (but don’t press enter yet!) [replace old.server.ip with Old Server IP and replace the date in .tar file]

1
scp -oStrictHostKeyChecking=no [email protected]:/home/backup/admin.2016-10-30.tar /home/backup/

Copy the root password the Old Server (Ctrl + C) to your clipboard

Ready?

Execute the “v-backup-user admin” command on the Terminal 1 Old Server
Wait a few seconds until the backup is done, or minutes if it’s a big website.
As soon as the backup is done, execute the scp “-oStrictHo….” on Terminal 2 New Server
Paste the password for the old server and the migration will begin.

1
admin.2016-10-29.tar                          100%   15MB  14.8MB/s   00:01

As soon as it’s done, in the same Terminal (Terminal 2 New Server) execute:

1
v-restore-user admin admin.2016-10-30.tar

Wait a few seconds until the backup is restored, or minutes if it’s a big website.

Go back to Cloudflare and change the IP to the new server.

This will immediately migrate everything, with zero downtime.

 

 

 

 

 

 

 

CentOS 7 .onion Tor Website Auto-install (2016)

THIS GUIDE IS INCOMPLETE, PLEASE USE THE TOR VESTACP GUIDE FOUND HERE.

The following guide is a simplified version of various “onion site guides” that are currently available. Guides from 2014 and 2015 seem to be quite lacking in substance and some use apache, phpmyadmin

This guide was written to make a tor version of a clearnet website that I operate so that tor users can access the website via tor. First a small foreword to explain the tools being used in this guide:

  • Nginx in a highly flexible and architecturally secure web server that will display web pages to your visitors. If you put an .html file in the folder that nginx shows to the internet, people will see that file.
  • Fail2ban is a program that bans IPs that try to connect to your server (the IP address). It is usually the first program you should install on any server because if you leave a new server on without fail2ban, even overnight, someone may already have your root password. On CentOS 7 it will tell you how many people have tried to log in since you last logged in.
1
2
3
4
[email protected]:~$ ssh [email protected]
Last failed login: Wed Oct 12 18:51:02 UTC 2016 from 32.11.23.32 on ssh:notty
There were 21211 failed login attempts since the last successful login.
Last login: Sun Oct 5 13:00:11 2016 from 59.34.59.34
1
yum update -y

# add epel repository if not already there

1
2
wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-8.noarch.rpm
rpm -ivh epel-release-7-8.noarch.rpm
1
yum install sudo -y
1
adduser nginx
1
passwd nginx
1
echo "%nginx ALL=(ALL:ALL) ALL" >> /etc/sudoers

su nginx

# update, install nginx, fail2ban and tor

sudo yum update -y
sudo yum install nginx -y
sudo yum install fail2ban -y
sudo yum install tor -y

exit

# start nginx

service nginx start

# remove torrc default file and make a new one with the following

rm -f /etc/tor/torrc

echo “HiddenServiceDir /var/lib/tor/hidden_service/” >> /etc/tor/torrc
echo “HiddenServicePort 80 127.0.0.1:80” >> /etc/tor/torrc

# start tor

service tor start

# show onion address

cat /var/lib/tor/hidden_service/hostname


You should turn off clearnet browsing by editing the following:

1
vi /etc/nginx/nginx.conf

Change:

1
2
3
4
5
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;

TO:

1
2
3
4
5
server {
listen 127.0.0.1:80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;

Create files for your website in here:

1
cd /usr/share/nginx/html/

For custom .onion URLs:

Follow this guide and then edit the following:

1
cd /var/lib/tor/hidden_service/

Replace hostname with your custom hostname:

1
vi hostname

Replace private_key with your new private key:

1
vi private_key

They should both be owned by toranon and both be chmod 600.

1
2
3
4
5
chmod 600 hostname
chmod 600 private_key
chown toranon:toranon -R *
service tor stop
service tor start

“Securely” generate Custom or Vanity .onion on VPS (Debian/Ubuntu/CentOS)

Word of warning: if you’re using a VPS service, such as Vultr, they may allow you to overclock your CPU which may be a terms of service violation as you’re using shared resources. On our first trial run, the CPU hit 1000% which may get you banned. Even on a dedicated instance we still hit 800% CPU usage. Final run hit 4000% on a 24x CPU instance, which, according to Vultr support, should max out at 2400% CPU usage.

Vultr cpu maximum usage

Security: since the private key is very, very private, you’ll want to use a server with a SATA drive, rather than an SSD, so you can wipe the drive when you’re finished.

Install Shallot .onion tor address generator on Debian & Ubuntu

On Debian 8 install libssl-dev.

1
apt-get install libssl-dev

# install git & make, and gcc

1
apt-get install git make gcc

# clone Shallot

1
git clone https://github.com/katmagic/Shallot.git

# enter the Shallot director

1
cd Shallot

# configure and make shallot

1
./configure && make

Install Shallot .onion tor address generator on CentOS

On CentOS 6 or 7 install openssl-devel.

1
yum install openssl-devel

# install git & make, and gcc

1
yum install git make gcc

# clone Shallot

1
git clone https://github.com/katmagic/Shallot.git

# enter the Shallot director

1
cd Shallot

# configure and make shallot

1
./configure && make

Running Shallot on your VPS

Note: press Ctrl + C to cancel generating an address at any time.

Running Shallot vanity tor address generator on a VPS

Now you can run shallot like this example:

1
./shallot ^test

IMPORTANT: if you’re generating addresses more than 4 characters long, you’ll want to use an output file. The following command will write the private key to a file when it’s finished. This is because if the address takes a long time to generate, you can come back to your server later when it’s finished.

If you lose connection without cancelling Shallot first, log back into your server and reboot it. VPS providers will be very unimpressed if you use 1000% of CPU power for a few hours.

1
./shallot ^test > address.txt

The scrip will go about its work. When it’s finished cat the contents of address.txt

1
cat ~/Shallot/address.txt

Example output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
-----------------------------------------------------------------
Found matching domain after 2688255 tries: testf46n4iedaarq.onion
-----------------------------------------------------------------
-----BEGIN RSA PRIVATE KEY-----
MIICWwIBAAKBgQC/TBokM3NXE+vTsJV+nTae1GTsr4/mwnHn6KxkWKWMZ4dvaV/B
EDqzfdJztQ5BylB37/N9nEAUuB0l9we4t1NEypine/uTmdEenJdy1zUrBA7sIRvS
gNJc2LzM5oY2HFqSp/LgJ/mYgdWGAxAYOjhsD0E4+GNa69+Gd3Squ/ro8QIDC7Gx
AoGALDSRySHbhHC9KQf8A+CKnOzLnjQXgFK6+X8bHnJiPtNjDCpCVXzNFQl/LcJl
Rauv//8/Y5dnrktAIMS1cCyoGfkBUNzlraZgGsRpnMkf3CU0GgxWTzGbAcPLJ9Zq
rqmntLGMs9SPEokxSBHLCF4WkQY94KtPfJ11AlaWp7kf03UCQQDgdBaq+/eDzgTh
yYaqhIYAIjNN5hApX/qn1yTVd/dw3wAq8fSjG6zApFJoZjqlgRBZ592HZAgU1Kev
GCctt7ZFAkEA2i8K3Sc5c+40/IkobJML0aeg40azjfo/mJvrc/q2FgodqGGKB4Jc
e09SBN7BR+7Kc9v9vFo4/feWzS9NBVV4vQJAQAgdmhtJGpZ/JHpPyALOHJ+hZiIC
AlOnRv9cMcB5rY4PfyY2KGUcV4KlXY8DHA6JtfPXVaqSeoaUTD/k1QjS9QJAC4IA
xf8L5o2LC9U353lCPz4MhQb6QN111w0tBbD4Lc3oCxMGrr5M1MVaE3MsZQU8pkbp
GNTWwraPf7WtNuMcuQJADCAO4Z5kXL5tUP4m2JRPIOAkiSa284iso736DcMBAP/3
+xIFaioKgFfOkvfCV/J8zbCFmGPE8NtdATDqQmx32Q==
-----END RSA PRIVATE KEY-----

Secure erasing the file when you’re finished.

If you’re using a SATA drive, first install secure-delete

Debian/Ubuntu:

1
apt-get install secure-delete

CentOS 6 or 7:

1
yum install srm


Then securely delete the file with srm

1
srm address.txt

The following command will help wipe a SATA disk driver. It will create a new file called nosuchfile and fill it with “zeros” until it fills the drive up completely and then it will delete the file.

Run this 5-6 times to be sure:

1
cat /dev/zero >nosuchfile; rm nosuchfile

If you chose a big server, you may have to wait a long time for the drive to wipe completely.


As per Shallot’s readme page,

“Time to Generate a .onion with a Given Number of Initial Characters on a 1.5Ghz Processor”

characters time to generate (approx.)
1 less than 1 second
2 less than 1 second
3 less than 1 second
4 2 seconds
5 1 minute
6 30 minutes
7 1 day
8 25 days
9 2.5 years
10 40 years
11 640 years
12 10 millenia
13 160 millenia
14 2.6 million years

 

Find all files matching pattern recursively linux (Debian/CentOS/Ubuntu)

I love grep and I love simple code.

1
find

find will show all files recursively from what directory you are currently in.

This will find all files, and then from that list it will show you files matching file.html

1
find | grep file.html

This will find all files again, and then grep files from that list matching file1.html OR file2.html

1
find | grep 'file1.html\|file2.html'

This will find all files again, and then grep files from that list matching file1.html OR file2.html and write the list to list.txt

1
find | grep 'file1.html\|file2.html' > list.txt

How To Find All IPs for VPN Service (Private Internet Access)

I wanted to use every Private Internet Access (PIA) IPs to run a script using each of their full list of IP addresses once.
For other companies, copy and paste a list of their whole network addresses list. For PIA, they

First, save a copy of all their servers names. For PIA I had to expand each section and then copy and paste all of the html into a text document or spreadsheet.

Private internet access IP list
When pasting into a spreadsheet, make sure to right click and select paste special and then unformatted text.

Private internet access server list

We want only the hostnames, so copy the column containing all the URLs and use the remove lines not containing tool to remove lines not containing:

1
.privateinternetaccess.com

Remove lines not containing privateinternetaccess.com

Save this list somewhere (doc/txt files).

Now use the prefix & suffix lines tool to prefix each line with

1
host

and suffix each line with

1
 >> ip-list.txt

Bulk find ip addresses of website

This will add each of the results of ‘host xxx.com’ to a file called ip-list. We use >> instead of > as a double right arrow adds the output to the file whereas a single right arrow will overwrite it.

Paste everything into a terminal and when it’s finished you should have a file with all of the IP addresses.

To clean the file up, find and replace ‘has address ‘ with a tab.

Replace has address with a tab

Paste this into a spreadsheet and you will have a column with all of the IP addresses.