How to Add Private Networking in CentOS 8/Ubuntu on Vultr (One-liners/AUTO)

Vultr uses ens7 as the private network interface name.

Your instances must be in the same location.

How to add private networking to Ubuntu in Vultr (netplan):

# centos 8

PRIVATE_IP='10.39.100.4'

echo 'TYPE="Ethernet" 
DEVICE="ens7" 
ONBOOT="yes" 
BOOTPROTO="none" 
IPADDR='"${PRIVATE_IP}"'
PREFIX=16
MTU=1450' > /etc/sysconfig/network-scripts/ifcfg-ens7

nmcli con load /etc/sysconfig/network-scripts/ifcfg-ens7
nmcli con up 'System ens7'


How to add private networking to Ubuntu in Vultr (netplan):

# ubuntu 18.04+

PRIVATE_IP='10.39.100.3'

MAC_ADDR=$(ifconfig -a | grep ens7 -A10 | grep -m1 -Eo '([0-9a-f]{2}:){5}[0-9a-f]{2}')

echo "network:
  version: 2
  renderer: networkd
  ethernets:
    ens7:
      match:
        macaddress: "${MAC_ADDR}"
      mtu: 1450
      dhcp4: no
      addresses: ["${PRIVATE_IP}"/16]"  >  /etc/netplan/10-ens7.yaml

netplan apply


Advanced Django Tricks: Dynamically & Automatically Generate Form Fields Using Python Local Variables

The following is a trick I use to dynamically generate form fields before you know the form column names.

dynamically generate django forms using python locals
dynamically generate django forms using python locals

This allows you to:

– dynamically populate form fields based on database columns
– editing database columns will automatically & dynamically add or remove the columns from your form

Brief overview:

OLD CODE:

class UserExtras(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE, primary_key=True)
    name = models.CharField(max_length=100)
    age = models.CharField(max_length=100)
    website = models.CharField(max_length=100)
    height = models.CharField(max_length=100)
    weight = models.CharField(max_length=100)
    shoe_size = models.CharField(max_length=100)

NEW CODE:

class UserExtras(models.Model):
    db_columns = ['name', 'age', 'website', 'height', 'weight', 'shoe_size']
    user = models.OneToOneField(User, on_delete=models.CASCADE, primary_key=True)
    for db_col in db_columns:
        if db_columns == 'index' or db_columns == 'id':
            continue
        locals()[db_col] = models.CharField(max_length=100)

What we are doing it adding the fields dynamically to the local variables inside our object

We can skip this by calling the table’s fields as these strings. Because these model objects do not include self, they cannot use setattr.

However, you can manually add them to locals()[array].

Consider this additional django form multiple fields example:

models.py

from django.db import models
from django.contrib.auth.models import User
from django.db import connections
from django.conf import settings

class UserTop(models.Model):
    email = models.CharField(max_length=100, blank=True, null=True)
    first_name = models.CharField(max_length=100, blank=True, null=True)
    last_name = models.TextField(blank=True, null=True)
    class Meta:
        db_table = 'auth_user'

views.py

from django.forms import *
from app.models import *
from .forms import *

def user_top(request):
    msg = None
    if request.method == 'POST':
        form = UserUpdateForm(request.POST)
        # id = UserTop.objects.get(id=request.user).user
        if form.is_valid(): 
            id = form.cleaned_data.get('id')
            email = form.cleaned_data.get('email')
            first_name = form.cleaned_data.get('first_name')
            last_name = form.cleaned_data.get('last_name')
            p = UserTop(
                id=id,
                email=email,
                first_name=first_name,
                last_name=last_name
            )
            p.save()
            msg = 'Saved.'
            return HttpResponse(json.dumps({'msg': msg}), content_type='application/json')
        else:
            msg = 'Form is not valid'
    else:
        form = ProfileForm()
    return render(msg, 'profile.html', {'form': form})

forms.py

class UserUpdateForm(forms.Form):
    id = forms.CharField(widget=forms.TextInput(attrs={"placeholder" : "Id","class": "form-control"}))
    email = forms.EmailField(widget=forms.EmailInput(attrs={"placeholder" : "Email","class": "form-control"}))
    first_name = forms.CharField(widget=forms.EmailInput(attrs={"placeholder" : "First Name","class": "form-control"}))
    last_name = forms.CharField(widget=forms.EmailInput(attrs={"placeholder" : "Last Name","class": "form-control"}))
    class Meta:
        model = Userfields = ('username', 'email', 'first_name', 'last_name')

urls.py

    path('edit-user-master/', views.user_top, name='profile.html'),

In this example we have a working custom form that allows a user to change his/her username, email, first_name and last_name.

Note the class Meta: db_table = ‘auth_user’ reference on the models.py. We can use forms.ModelForm but when we use forms.Form we must add the class Meta table name.

Magic Django Automatic Dynamic Form Field Generator

ADD TO THE TOP OF models.py

def KeyPairs(arg):
    from django.db import connection
    import pandas as pd
    cursor = connection.cursor()
    df = pd.read_sql_query(
        "SELECT TABLE_NAME, COLUMN_NAME FROM `information_schema`.`COLUMNS` \
        WHERE CONVERT(`TABLE_NAME` USING utf8) LIKE '%"+arg+"%'",
        connection
    )
    return df

This function method will query your Database a return a pandas dataframe of your Column Names.

Now consider this replacement to the class UserTop(models.Model) at the start of this demonstration:

models.py

class UserTop(models.Model):
    user = models.OneToOneField(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, primary_key=True)
    key_pairs = KeyPairs('auth_user')
    kp_list = key_pairs.COLUMN_NAME.to_list()
    itemy = ''
    for column_name in kp_list:
        # set columns to ignore here
        if column_name == 'email' or column_name == 'user_id':
            continue
        if column_name == 'bio':
            locals()[itemy] = models.TextField(blank=True, null=True)
        else:
            locals()[itemy] = models.CharField(max_length=100)
    class Meta:
        db_table = 'auth_user'

Add many 200+ Form Fields automatically in Django go in this example:

THIS:

django large forms automatically generate fields before
django large forms automatically generate fields before

BECOMES THIS!!

models.py

def KeyPairs(arg):
    from django.db import connection
    import pandas as pd
    cursor = connection.cursor()
    df = pd.read_sql_query(
        "SELECT TABLE_NAME, COLUMN_NAME FROM `information_schema`.`COLUMNS` \
        WHERE CONVERT(`TABLE_NAME` USING utf8) LIKE '%"+arg+"%'",
        connection
    )
    return df

class Triggers(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE, primary_key=True) #, related_name='user'
    key_pairs = KeyPairs('app_triggers')
    kp_list = key_pairs.COLUMN_NAME.to_list()
    itemy = ''
    for itemy in kp_list:
        if itemy == 'index' or itemy == 'id':
            continue
        locals()[itemy+'_bool'] = models.BooleanField(max_length=100)
        locals()[itemy+'_range'] = models.SmallIntegerField()
        locals()[itemy+'_method'] = models.CharField(max_length=100)

Just to demonstrate how powerful this is, I used it to add 200+ form fields.

dynamically generate django forms using python locals
dynamically generate django forms using python locals

PIPENV Crontab & Docker: How to Run PIPENV Python Automatically Properly in Cron Jobs with Pipenv – Running Django On a Server

How to run Pipenv in a Docker Container, or using Crontab

When you run pipenv shell, you will notice a link in you console terminal output with a dot (.) symbol.
This symbol is the shell command for “source”.

pipenv cron job source activate
pipenv cron job source activate

If you haven’t installed pipenv here is the setup:

# Install Pipenv on Ubuntu / Debian (Works on Docker)
apt install pipenv -y
pip3 install --upgrade pip
pip3 install --upgrade pipenv
touch Pipfile
export SHELL=/bin/bash
python3 -m pipenv --python 3.8

# CentOS / Red Hat
yum install epel-release
yum install python3 python3-pip
pip3 install pipenv
touch Pipfile
pipenv install

Locate your activate bin file:

find $HOME/.local/share/virtualenvs/ | grep 'activate$'

Now you can use that activate bin inside your cron jobs as follow:

# these two commands are the same
. /home/admin/.local/share/virtualenvs/private-3Qz0_VCC/bin/activate
# or
source /home/admin/.local/share/virtualenvs/private-3Qz0_VCC/bin/activate

This gets your inside your python pipenv shell, where running “python” does it’s magic.

For example, here’s how to run Django
Type “crontab -e” and enter the cron job

source /home/admin/.local/share/virtualenvs/private-3Qz0_VCC/bin/activate && cd /home/user/ && nohup python manage.py runserver 127.0.0.1:9988 &

Here are some examples to help you understand

Here’s a fully automated Docker way:

# in any ubuntu image

RUN apt update -y
RUN apt install pipenv -y
RUN pip3 install --upgrade pip
RUN pip3 install --upgrade pipenv
WORKDIR /opt
RUN export SHELL=/bin/bash
RUN python3 -m pipenv --python 3.8

If you receive this error:

  File "/usr/lib/python3/dist-packages/pipenv/project.py", line 475, in create_pipfile
    config_parser = ConfigOptionParser(name=self.name)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/pip9/baseparser.py", line 149, in __init__
    assert self.name

AssertionError

you can try two things

export SHELL=/bin/bash
pipenv shell
touch pipfile
pipenv shell

How to Convert SQLite to MySQL & Connect DJANGO to WEB + MySQL Server: Converting a Django sqlite3 to MySQL

If you’ve been using Django, you know that the migrate function creates a locally stored sqlite database. But what if you want to upload your website to the internet and use a MySQL database?

In this example I will use an AppSeed template.

# skip this if you already have your django website
clone https://github.com/app-generator/django-dashboard-dattaable.git
cd django-dashboard-dattaable
pipenv shell
pip install -r requirements.txt
python manage.py makemigrations
python manage.py migrate
python manage.py runserver
Django Move Website To Internet
Django Move Website To Internet

The website should now be at localhost:8000.

But how do you move this website to the Internet?

You need a server and to convert the SQlite db into MySQL. It is available on most OS’es:

Install DB Browser for SQLite

brew cask install db-browser-for-sqlite # MAC
sudo pacman -S db-browser-for-sqlite # ARCH
sudo apt install sqlitebrowser # DEBIAN UBUNTU
yum install sqlitebrowser # CENTOS RHEL

Inside your Django project folder, open db.sqlite3 using the SQLite browser.

File > export > Database to SQL File

Sqlite to MySQL Django Convert Database
Sqlite to MySQL Django Convert Database

Open the new file called db.sqlite3.sql inside a TEXT EDITOR, like VSCode or Sublime Text

You need to replace exactly the follow things:


Replace this column


with this column

[cc]”[/cc] [cc]'[/cc]
[cc]AUTOINCREMENT[/cc] [cc]AUTO_INCREMENT[/cc]
[cc]bool[/cc] [cc]boolean[/cc]
[cc]DEFERRABLE INITIALLY DEFERRED[/cc] [cc] [/cc]
[cc]BEGIN TRANSACTION;[/cc] [cc]START TRANSACTION ;;[/cc]
[cc]INSERT INTO[/cc] [cc]INSERT IGNORE INTO[/cc]
[cc]COMMIT;[/cc] [cc];;[/cc]

Before you paste into SQL you must have at least 1 user registered on your local website. This is required for the foreign key checks!

Now you can import your Django SQLite3 Database into MySQL!
Use PhpMyAdmin or use the command line MySQL by logging into your server as root and pasting the whole block at once.

Next you must add the new database to your settings.py inside your Django Project:

Replace the existing django.db.backends.sqlite3 entry in your settings.py:

# Database
# https://docs.djangoproject.com/en/3.0/ref/settings/#databases
DATABASES = {
    # 'default': {
    #     'ENGINE': 'django.db.backends.sqlite3',
    #     'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    # },
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'admin_newdb',
        'USER': 'admin_userace',
        'PASSWORD': 'abc22829md',
        'HOST': 'xxx.xxx.xxx.xxx',
        'PORT': '3306'
    }
}

Edit __init.py__ inside the SAME FOLDER AS settings.py and add:

import pymysql
pymysql.install_as_MySQLdb()

Open your project directory requirements.txt file and add pymysql

pip install -r requirements.txt
# or 
pip install pymysql

Now you can connect to the external Database instead of the local SQLite3 Database! Magic, right?

If you don’t have a server yet or a fresh MySQL database (table) ready, then keep following this tutorial:

Uploading Your Django Website Online

Sign up to Vultr.com and launch any of these:

Ubuntu 18.04 with 1 CPU and 1GB of RAM

CentOS 7 (without SE Linux) with 1 CPU and 1GB of RAM

Vultr Host Django Website Tutorial How To Install Ubuntu 18.04 Django And CentOS Django
Vultr Host Django Website Tutorial How To Install Ubuntu 18.04 Django And CentOS Django

Once the server is “ready”, open a Terminal or Putty

ssh [email protected]
# install VestaCP
curl -O http://vestacp.com/pub/vst-install.sh
bash vst-install.sh

Once that’s finished, go to the URL and login with the password displayed in the Terminal. https://xxx.xxx.xxx.xxx:8083

Go to Databases tab in the admin panel and create a new database and password. Put those details into your settings.py

django mysql database hosting online
django mysql database hosting online

Open https://xxx.xxx.xxx.xxx/phpmyadmin and login with those details your just created.

Click on your Database, upload the database, TURN OFF FOREIGN KEY CHECKING WHILE UPLOADING.

uploading django database to website
uploading django database to website

One Liner: ULTIMATE Kubernetes Install! K3s! Ubuntu, CentOS, Debian, RHEL. Best k8s, minikube, okd, Openshift Alternative.

Forget minikube, k8s, docker swarm, okd, Openshift or any other bloated mess of amateur shell scripts.

This is the one. Install Docker on ANY of the above Operating Systems and then run:

# run this and you're done!
curl -sfL https://get.k3s.io | sh -
kubectl get node
kubectl get get all --all-namespaces

TIP: instead of writing kubectl all the time, it’s much more efficient to put an alias for this command:

echo 'alias ks=kubectl' >> ~/.bashrc
echo 'alias k=kubectl' >> ~/.bashrc
source ~/.bashrc

Now you can type ks or just k to run kubectl commands. I like ks because it sits right under the fingers, ready for action.

If you haven’t got Docker, run this first:

Install Docker CE on Ubuntu

apt-get remove docker docker-engine docker.io containerd runc -y
apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg |  apt-key add -
apt-key fingerprint 0EBFCD88
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update -y
apt-get install docker-ce docker-ce-cli containerd.io -y
docker run hello-world
curl -sfL https://get.k3s.io | sh -

And you’re done!

Install a Local Docker Registry On your Kubernetes VM

Put the docker file in your current directory

docker run -d -p 5000:5000 --restart=always --name registry registry:2
docker build --no-cache -t IMAGENAME .
docker tag IMAGENAME localhost:5000/IMAGENAME
docker push localhost:5000/IMAGENAME

Now you can pull direct from that image

docker run localhost:5000/IMAGENAME

Install the Kubernetes Dashboard on K3s

GITHUB_URL=https://github.com/kubernetes/dashboard/releases
VERSION_KUBE_DASHBOARD=$(curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')
sudo k3s kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${VERSION_KUBE_DASHBOARD}/aio/deploy/recommended.yaml

echo 'apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard' > dashboard.admin-user.yml

echo 'apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard' > dashboard.admin-user-role.yml

k3s kubectl create -f dashboard.admin-user.yml -f dashboard.admin-user-role.yml
k3s kubectl -n kubernetes-dashboard describe secret admin-user-token | grep ^token
k3s kubectl proxy

Linux/Mac: Ffmpeg render/convert or export video using AMD GPU Graphics Card in Arch Linux, Mint, Debian, Manjaro

To use your GPU rather than your CPU to convert videos under AMD architecture, you need to use va-api, video acceleration API.

Results:
51 seconds using CPU 100fps 21mb file-size
34 seconds using GPU 130fps 42mb file-size

Requirements:

# VA-API for Radeon HD 2000 and newer require libva-mesa-driver
# VDPAU for Radeon R300 and newer require mesa-vdpau
# libva-vdpau-driver
# you can just install both

# arch manjaro
sudo pacman -S libva-mesa-driver mesa-vdpau libva-vdpau-driver

# debian ubuntu
apt install mesa-va-drivers -y

You may need to reboot in order for the kernel to load with the new updates.

This will install a library a library for

After installing libva vdpau drivers for mesa, you will have a device driver that you can reference in ffmpeg /dev/dri/renderD128

Convert ffmpeg files using your AMD Graphics Card, Laptop Radeon Vega Mobile works too.

Initialize the device

# init device
ffmpeg -init_hw_device vaapi=gpu:/dev/dri/renderD128

Now you can convert with the GPU:

IN_VIDEO=video1.mp4
OUT_VIDEO=video2.mp4
ffmpeg -i "$IN_VIDEO" -vaapi_device /dev/dri/renderD128 -vcodec h264_vaapi -vf format='nv12|vaapi,hwupload' "$OUT_VIDEO"

How to tell if my GPU is being used to convert videos using the gpu?

Run two tests.

# test 1
ffmpeg -i in.mp4 normal.mp4

# test 2
ffmpeg -i in.mp4 -vaapi_device /dev/dri/renderD128 -vcodec h264_vaapi -vf format='nv12|vaapi,hwupload' accelerated.mp4
cpu ffmpeg high usage
cpu ffmpeg high usage
amd video gpu ffmpeg video convert
amd video gpu ffmpeg video convert

The GPU accelerated version using AMD Ryzen 5 3500U with Radeon Vega finished before we could even get a decent CPU reading!

Add Swap RAM on CentOS 8 (one line, automatically)

Paste the following code into a terminal on CentOS 8. It also works on CentOS 7 and will work on Red Hat 8, RHEL8, as well.

Where the first line has 4G, you can increase or decrease this number which refers to the size of the Swap RAM partition.

fallocate -l 4G /swapfile
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
swapon -s
echo '/swapfile swap swap sw 0 0' >> /etc/fstab

Navigate go up one directory in Python using paths.

import os
from os.path import basename
path = "."
path = os.path.realpath(path)
path = path[:-len(basename(path))]

If you want to know what this does:
path starts as

.

then os.path.realpath(arg) turns it into

/abd/def/ghik

which is the current working directory.
Now path is a string: “/abd/def/ghik”

basename(path)

will give us “ghik” which is a 4 letter word.
The directory above this directory is the realpath -4 letters, from the end.
We can use [from:to] to select a range of items in a python list.
So we use from 0 to -4

path[:-len(basename(path))]

If you want to remove the last “/”, use the range 0:-5

path = path[:-(len(basename(path))+1)]