Objetivo

Descrever a instalação de configuração de um servidor GANETI em DEBIAN, abordando desde o tipo de RAID, particionamento de disco e configuração completo de um GANETI com hypervisor XEN e eventualmente KVM.

Especialistas Envolvidos

Guto Carvalho
Daniel Sobral

Ambiente

Podemos ter vários tipos de máquinas atuando como nós GANETI, vamos abordar cada cenário nesta wikipage.

Dell PowerEdge R900

RAID

  • Use RAID '0', entre no configurador do raid durante o boot.

particionamento

  • Criar partição / com 12 GB
  • Criar partição SWAP com 2 GB
  • Criar partição LVM com o restante do espaço em disco, nome do volume xenvg

Dell PowerEdge 2950

Debian

Atualmente nosso ambiente é todo LENNY porém em pouco tempo o SQUEEZE será o novo stable, portando em breve a seção do SQUEEZE será criada.

Squeeze

Reservado para configuração no futuro STABLE.

Lenny

Sobre instalação do Debian

Arquitetura: 64 Bits
Tipo de instalação: netinstall

módulos necessários

Para instalar o Debian em um R900 serão necessários os módulos da HBA e da placa Gigabit.

Uma forma de instalar é manter estes pacotes no mesmo pendrive que você usou para instalar o Debian, após o primeiro boot istale os pacotes com o DPKG e reinicie novamente.

# dpkg -i firmware-qlogic.deb
# dpkg -i firmware-bnx2.deb

hba

Instalar o modulo firmware-qlogic

Faça o download do pacote em http://packages.debian.org/lenny/all/firmware-qlogic/download

ethernet gigabit

Instalar o modulo firmware-bnx2

Faça o download o pacote em http://packages.debian.org/lenny/all/firmware-bnx2/download

atualizando o debian

configure os mirrors, ajuste os fontes

sources.lists
deb http://ftp.br.debian.org/debian/ lenny main non-free contrib
deb http://security.debian.org/ lenny/updates main contrib non-free
deb http://volatile.debian.org/debian-volatile lenny/volatile main contrib non-free
deb http://backports.debian.org/debian-backports lenny-backports main

se nao existir crie o arquivo /etc/apt/apt.conf.d/80Default para garantir que só vai usar pacotes da stable para atualizar.

80Default
APT::Default-Release "stable";

agora vamos atualizar, perceba que comentei o backports, vamos descomentar mais na frente

# aptitude update && aptitude safe-upgrade -y

instalando pacotes básicos

# aptitude install vim openssh-server ntpdate zabbix-agent host locate screen sudo less udev lsb-release bzip2

Ganeti

O Ganeti é um software de gerencia de máquinas virtuais em Cluster.

Ele suporta hoje gerenciar hypervisor XEN e KVM.

Squeeze

Espaço reservado para configuração na futura STABLE do debian.

Lenny

Aqui vamos instalar tudo em arquitetura 64 bits

ajustando hosts

hosts
127.0.0.1       localhost
192.168.21.16   gnt-node06.empresa gnt-node06
hostname
gnt-node06.empresa

execute

/etc/init.d/hostname.sh start

instalando o XEN

# aptitude install xen-linux-system-2.6.26-2-xen-amd64

configurando o xen

/etc/xen/xend-config.sxp
...
(xend-relocation-server yes)
(xend-relocation-port 8002)
(xend-relocation-address '')
(xend-relocation-hosts-allow '')
...
(vnc-listen '0.0.0.0')
(vncpasswd 'senha')
...

usar a mesma configuração dos demais nodes do cluster.

ajustando o GRUB

/boot/grub/menu.lst
...
## Xen hypervisor options to use with the default Xen boot option
# xenhopt=dom0_mem=1024M
 
## Xen Linux kernel options to use with the default Xen boot option
# xenkopt=console=tty0 maxcpus=1
...

Preste muita atenção no parâmetro xenhopt=dom0_mem=1024M o M significa MegaBytes, sem isso a máquina nao boota.

Nos parâmetros xenhopt e xenkopt existe o '#' antes do parametro, nao retire.

A documentação recomenda colocar nosmp no xenkopt mas isso nao funciona. O maxcpus=1 tem o mesmo efeito.

La na opção de inicialização do Kernel XEN

altere

kernel          /xen-3.2-1-amd64.gz

para

kernel          /xen-3.2-1-amd64.gz dom0_mem=1024M

Agora precisamos atualizar as entradas do grub

# update-grub

Precisamos criar 2 symlinks no /boot

ln -s vmlinuz-2.6.26-2-xen-amd64 vmlinuz-2.6-xenU
ln -s initrd.img-2.6.26-2-xen-amd64 initrd.img-2.6-xenU

A instrução do site do GANETI indica que o segundo link seria initrd-2.6-xenU, mas faltou o '.img'

instalando o DRBD

antes de iniciar a configuração reboot o sistema para ele entrar no kernel xen

instalando pacotes source do drbd e compilando via m-a

# aptitude install drbd8-source drbd8-utils
# m-a update
# m-a a-i drbd8

configurando o sistema para iniciar o drbd no boot

# echo drbd minor_count=128 usermode_helper=/bin/true >> /etc/modules
# depmod -a
# modprobe drbd minor_count=128 usermode_helper=/bin/true

configurando o DRBD

salvando arquivo padrao de configuracaodo drbd

# mv /etc/drbd.conf /etc/drbd.conf.orig

agora crie o arquivo drbd conforme abaixo

/etc/drbd.conf
#
# drbd.conf example
#
# parameters you _need_ to change are the hostname, device, disk,
# meta-disk, address and port in the "on <hostname> {}" sections.
#
# you ought to know about the protocol, and the various timeouts.
#
# you probably want to set the rate in the syncer sections
 
#
# NOTE common pitfall:
# rate is given in units of _byte_ not bit
#
 
#
# increase timeout and maybe ping-int in net{}, if you see
# problems with "connection lost/connection established"
# (or change your setup to reduce network latency; make sure full
#  duplex behaves as such; check average roundtrip times while
#  network is saturated; and so on ...)
#
 
skip {
  As you can see, you can also comment chunks of text
  with a 'skip[optional nonsense]{ skipped text }' section.
  This comes in handy, if you just want to comment out
  some 'resource <some name> {...}' section:
  just precede it with 'skip'.
 
  The basic format of option assignment is
  <option name><linear whitespace><value>;
 
  It should be obvious from the examples below,
  but if you really care to know the details:
 
  <option name> :=
        valid options in the respective scope
  <value>  := <num>|<string>|<choice>|...
              depending on the set of allowed values
              for the respective option.
  <num>    := [0-9]+, sometimes with an optional suffix of K,M,G
  <string> := (<name>|\"([^\"\\\n]*|\\.)*\")+
  <name>   := [/_.A-Za-z0-9-]+
}
 
#
# At most ONE global section is allowed.
# It must precede any resource section.
#
global {
    # By default we load the module with a minor-count of 32. In case you
    # have more devices in your config, the module gets loaded with
    # a minor-count that ensures that you have 10 minors spare.
    # In case 10 spare minors are too little for you, you can set the
    # minor-count exeplicit here. ( Note, in contrast to DRBD-0.7 an
    # unused, spare minor has only a very little overhead of allocated
    # memory (a single pointer to be exact). )
    #
    # minor-count 64;
 
    # The user dialog counts and displays the seconds it waited so
    # far. You might want to disable this if you have the console
    # of your server connected to a serial terminal server with
    # limited logging capacity.
    # The Dialog will print the count each 'dialog-refresh' seconds,
    # set it to 0 to disable redrawing completely. [ default = 1 ]
    #
    # dialog-refresh 5; # 5 seconds
 
    # You might disable one of drbdadm's sanity check.
    # disable-ip-verification;
 
    # Participate in DRBD's online usage counter at http://usage.drbd.org
    # possilbe options: ask, yes, no. Default is ask. In case you do not
    # know, set it to ask, and follow the on screen instructions later.
    usage-count yes;
}
 
 
#
# The common section can have all the sections a resource can have but
# not the host section (started with the "on" keyword).
# The common section must precede all resources.
# All resources inherit the settings from the common section.
# Whereas settings in the resources have precedence over the common
# setting.
#
 
common {
  syncer { rate 30M;
	   al-extents 3833; 
	}
}

ajustando interfaces

comente a configuracao inicial

#auto eth0
#iface eth0 inet static
#        address 192.168.21.16
#        netmask 255.255.254.0
#        network 192.168.20.0
#        broadcast 192.168.21.255
#        gateway 192.168.20.3

e insira a configuração

auto xen-br0
iface xen-br0 inet static
       address 192.168.21.16
       netmask 255.255.254.0
       network 192.168.20.0
       broadcast 192.168.21.255
       gateway 192.168.20.3
       bridge_ports eth0
       bridge_fd 9
       bridge_hello 2
       bridge_maxage 12
       bridge_stp off
       # dns-* options are implemented by the resolvconf package, if installed
       dns-nameservers 172.16.1.1
       dns-search empresa

reinicie a rede

       
# /etc/init.d/networking restart

verifique se a interface esta no ar

gnt-node06:~# ip a show xen-br0
10: xen-br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 00:15:17:cd:e9:dc brd ff:ff:ff:ff:ff:ff
    inet 192.168.21.16/23 brd 192.168.21.255 scope global xen-br0
    inet6 fe80::215:17ff:fecd:e9dc/64 scope link 
       valid_lft forever preferred_lft forever

verifique se esta em bridge com a interface correta

gnt-node06:~# brctl show xenbr0
bridge name	bridge id		STP enabled	interfaces
xen-br0		8000.001517cde9dc	no		eth0

checando o LVM

Como já criamos o LVM na instalação, vamos só conferir

gnt-node06:~# vgdisplay 
--- Volume group ---
VG Name               xenvg
System ID             
Format                lvm2
Metadata Areas        1
Metadata Sequence No  1
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                0
Open LV               0
Max PV                0
Cur PV                1
Act PV                1
VG Size               259.20 GB
PE Size               4.00 MB
Total PE              66356
Alloc PE / Size       0 / 0   
Free  PE / Size       66356 / 259.20 GB
VG UUID               eHUEBu-CXXO-bQVi-PHsO-dTKH-6MGz-JpdqQa

configurando o LVM

edite o arquivo lvm.conf em /etc e comente a linha abaixo

#filter = [ "a/.*/" ]

insira a linha abaixo

filter = [ "r|/dev/cdrom|", "r|/dev/drbd[0-9]+|" ]

instalando o GANETI

Habilite o lenny-backports em seu sources.list

atualize os indices de pacotes

# aptitude update

agora instale

# aptitude -t lenny-backports install ganeti2

confira se os pacotes abaixos foram instalados

# aptitude install debootstrap dump kpartx

se você ainda não reboot com o kernel XEN e DRBD, reboote agora.

criando ganeti master

em construção

criando ganeti node (NAO MASTER)

vá para o nó master, em nosso caso GNT-NODE01.

execute o comando abaixo para adicionar o nó ao cluster

# gnt-node add gnt-node06.empresa
  
-- WARNING -- 
Performing this operation is going to replace the ssh daemon keypair
on the target machine (gnt-node06.empresa) with the ones of the current one
and grant full intra-cluster ssh root access to/from it

The authenticity of host 'gnt-node06.empresa (192.168.21.16)' can't be established.
RSA key fingerprint is 2e:05:4c:54:1a:3a:b5:7b:89:1d:dc:78:0b:fc:8c:f0.
Are you sure you want to continue connecting (yes/no)? yes
root@gnt-node06.empresa's password: 
Thu Nov 25 11:31:47 2010  - INFO: Node will be a master candidate

verifique se o cluster está funcionando adequadamente

# gnt-cluster verify

Thu Nov 25 11:32:03 2010 * Verifying global settings
Thu Nov 25 11:32:03 2010 * Gathering data (6 nodes)
Thu Nov 25 11:32:06 2010 * Verifying node status
Thu Nov 25 11:32:06 2010 * Verifying instance status
Thu Nov 25 11:32:06 2010 * Verifying orphan volumes
Thu Nov 25 11:32:06 2010 * Verifying orphan instances
Thu Nov 25 11:32:06 2010 * Verifying N+1 Memory redundancy
Thu Nov 25 11:32:06 2010 * Other Notes
Thu Nov 25 11:32:06 2010   - NOTICE: 101 non-redundant instance(s) found.
Thu Nov 25 11:32:06 2010 * Hooks Results

que beleza, NENHUM erro hehehe… vamos ver todos os nós do cluster

# gnt-node list

Node           DTotal  DFree MTotal MNode  MFree Pinst Sinst
gnt-node01.empresa 272.1G   7.0G 128.0G  1.0G  58.9G    20     0
gnt-node02.empresa 272.0G   852M 128.0G  1.0G  58.3G    22     0
gnt-node03.empresa 784.1G 226.0G 128.0G  1.0G  52.8G    31     0
gnt-node04.empresa 262.9G  22.9G 128.0G  1.0G  40.3G    16     0
gnt-node05.empresa 262.9G  62.9G 128.0G  1.0G 115.3G    12     0
gnt-node06.empresa 259.2G 259.2G 128.0G  1.0G 125.3G     0     0

verifique se o diretório /etc/ganeti/instance-debootstrap/hooks está vazio na node06, se estiver, copie o arquivo

instance-debootstrap.tar.gz

que está em /etc/ganeti na gnt-node01 e substitua o diretório instance-debootstrap do - novo - node06.

Referências

Documentação oficial do Ganeti http://docs.ganeti.org/ganeti/2.2/html/install.html



ganeti_instalando_em_os_debian.txt · Last modified: 2011/01/04 00:38 by gutocarvalho
CC Attribution-Share Alike 4.0 International
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0