Fast Method to repair MySQL Master-Master Replication

Fast Method to repair Master-Master Replication

1. On th Good Server
a. mysqldump --add-drop-table --master-data  --quick -u root -p databaseName > databaseName.sql
b. bzip2 databaseName.sql
c.  scp databaseName.sql.bz2 -P port username@192.168.100.5:


2. On Bad Server
bunzip2 databaseName.sql.bz2

mysql> stop slave;
mysql -u root -p databaseName < databaseName.sql
mysql> start slave;
mysql> show slave status\G

You should see Slave_IO_Running: Yes and Slave_SQL_Running: Yes.

Now --
mysql> flush tables with read lock;
mysql> show master status;

Make a note of the File and Position rows.

mysql-bin.000044 | 11726351


3. On the Good Server
mysql> stop slave;
mysql> change master to master_log_file='mysql-bin.000044',
master_log_pos=11726351;
mysql> start slave;
mysql> show slave status\G

You should see Slave_IO_Running: Yes and Slave_SQL_Running: Yes.

4. on the Bad Server
mysql> unlock tables;

Setup Proxmox Cluster High Availability

from the main node issue the command

pvecm create clusterone 

from the other node :

pvecm add MAIN_NODE_IP 

we got cluster running now .

High Availability :

click on the 'Datacenter' --> HA --> Add : choose the virtual machine you need to add to HA.

to test HA is running  or not , go to the node where the virtual machine is install and down the network interface :

ifdown eth0 

the virtual machine should be still running while the main node is off.

Setting up Gluster
Reference 1
Reference 2
Refernce 3
Notice : Directory Need to be Manipulated to serve this job check Reference 2

 Add the GPG key to apt:



    wget -O - http://download.gluster.org/pub/gluster/glusterfs/3.11/rsa.pub | apt-key add -



Add the source:



    echo deb http://download.gluster.org/pub/gluster/glusterfs/3.11/LATEST/Debian/jessie/apt jessie main > /etc/apt/sources.list.d/gluster.list 



or:

    echo deb http://download.gluster.org/pub/gluster/glusterfs/3.11/LATEST/Debian/stretch/apt stretch main > /etc/apt/sources.list.d/gluster.list 



Update package list:



    apt-get update



Install:



    apt-get install  glusterfs-server glusterfs-client 

from the master node

  gluster peer probe node1.proxmox
 

create the share named testvol with two replicas (please note that the number of replicas is equal to the number of servers :

 gluster volume create testvol replica 2 transport tcp master.proxmox:/data/testvol node1.proxmox:/data/testvol force

Gluster may complainn of spli brain issue , but I think proxmox can avoid that using fencing or Watchdog. Start the Volume

gluster volume start testvol

you can check the connection is established or not using :

 netstat -tap | grep glusterfsd 

if not check the required ports are open .
Now , from Proxmox Datacenter node , Choose --> Storage --> Add --> GlusterFS enter the name you like for this gluster storage, the ip of the two servers and the volume name "testvol" in our case

Don't forget to enables watchdog

just uncomment the line

# select watchdog module (default is softdog)
WATCHDOG_MODULE=ipmi_watchdog

Now It is time to do security job to secure the system

Commands:
Start Stop the Gluster service

 service glusterfs-server start | stop | status 

Information about gluster Volume

gluster volume info

Check status of high availability

systemctl status pve-ha-crm.service 

Check the Cluster Status

 pvecm status 

High Availability Command:

USAGE: ha-manager  [ARGS] [OPTIONS]
       ha-manager groupadd  -nodes  [OPTIONS]
       ha-manager groupconfig
       ha-manager groupremove 
       ha-manager groupset  [OPTIONS]

       ha-manager add  [OPTIONS]
       ha-manager config  [OPTIONS]
       ha-manager migrate  
       ha-manager relocate  
       ha-manager remove 
       ha-manager set  [OPTIONS]

       ha-manager status  [OPTIONS]

       ha-manager help [] [OPTIONS]

Proxmox Network Issues

Setting Up Nat Network On Promox

Promox - Promox Cluster - High Availability -- what the difference

Promox alone is a Virtualization Software that enables you to run many Virtual Machines on a single Server Machine .
Proxmox-Cluster : Where you can manage many physical servers from  one point , each Server is running Proxmox and many Virtual machines.

Proxmox-Cluster-High-Availability: the Promox Cluster provide if a virtual machine (VM) is configured as HA and the physical host fails, the VM is automatically restarted on one of the remaining Proxmox VE Cluster nodes

proxmox cluster TASK ERROR: No accelerator found!

when trying to start VM I got en erroe with this message :

TASK ERROR: No accelerator found!

Solved By Disable "KVM hardware virtualizaion"

from options click the option titled  KVM hardware virtualizaion and disable it

Cannot migrate VM in proxmox cluster

ERROR: Failed to sync data - can't do online migration - VM uses local disks

iso removed from vm-100.conf

Folder is not shared, So I tried to cluster the volume group

vgchange -cy

Trying Again to migrate the Virtual machine gives this error:

Skipping clustered volume group pve
Cannot process volume group pve

rebooting will cause the system not running and the grub will hang on :initramfs

run the command

  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
  VG   #PV #LV #SN Attr   VSize  VFree
  pve    1   4   0 wz--nc 49.75g 6.07g

nc means clustered  So remove the clustering from the volume group:

vgchange -cn vgname --config 'global {locking_type = 0}'

that will bring back the pve lvm group to non clustered If the volume group is set to clustered (vgchange -cy vgname) then the volume group can only be activated if your Red Hat High Availability cluster is quorate, and Clustered LVM Daemon (CLVMD) is active. [Reference]

At last;
when you migrate a vm ; first shutdown the vm then press migrate option  and enable online checkbox .

Proxmox Cannot add a new node cluster not ready - no quorum?

For some reason when I tried to add a new node I got this error

cluster not ready - no quorum?

I tried a lot of the google suggested solutions but nothing  succeeded , at last I tried this trick

pvecm e 1

and that did the trick and solved the issue .

"Failed to start Corosync Cluster Engine"  on the second node for only two nodes setup proxmox :
the problem was in /etc/hosts --> wrong IP.

sudo permission on remote server using sftp

when connecting to a remote server using sftp command line , you may face a permission denied when you want to run commands on some folders . but you want to do that command using sudo permission ,So you just need to enter this command :

sftp -s "sudo /usr/lib/openssh/sftp-server" 188.40.xxx.xxx

Provided that the username is already has ALL permissions in the sudoers file .Ref

HG658 V2 Gateway destination host unreachable between LAN computers

On HG658 V2 Gateway when I tried to ping one of the PC's that are connected to this LAN the response always  the destination host unreachable , So after many hours of digging down to figure out the solution , it return that there is a checked  option on the Router  that should not be checked . this option called "Enable AP isolation" , AP (access point) isolation  or  'Client Isolation' or 'Guest Mode', The name depend on your router type .You can  find this option some where in the Wireless  LAN Properties, So disabling this option well allow computers to reach each other .