... | ... | @@ -164,36 +164,36 @@ the [storage servers documentation](howto/new-machine-cymru#storage-servers) for |
|
|
This assumes you are creating a 500GB VM, partitioned on the Linux
|
|
|
host, *not* on the iSCSI volume. TODO: change those instructions to
|
|
|
create one volume per partition, so that those can be resized more
|
|
|
easily. The following is how `web-chi-03` was setup.
|
|
|
easily. The following is how `tb-builder-03` was setup.
|
|
|
|
|
|
1. create the disk on the SAN and assign it to the host group:
|
|
|
|
|
|
create virtualDisk physicalDiskCount=3 raidLevel=5 userLabel="web-chi-03" capacity=500GB;
|
|
|
set virtualDisk ["web-chi-03"] logicalUnitNumber=4 hostGroup="gnt-chi";
|
|
|
puppet agent --disable "creating a SAN disk"
|
|
|
$EDITOR /usr/local/sbin/tpo-create-san-disks
|
|
|
/usr/local/sbin/tpo-create-san-disks
|
|
|
puppet agent --enable
|
|
|
|
|
|
WARNING: the above script needs to be edited before it does the
|
|
|
right thing. It will show the LUN numbers in use below. This,
|
|
|
obviously, is not ideal, and should be replaced by a Ganeti
|
|
|
external storage provider.
|
|
|
|
|
|
NOTE: the `logicalUnitNumber` here must be an increment from the
|
|
|
previous highest LUN. See also the [disk creation instructions](howto/new-machine-cymru#creating-a-disk)
|
|
|
for a discussion.
|
|
|
|
|
|
2. detect the new device on the Linux side:
|
|
|
|
|
|
iscsiadm -m node --rescan
|
|
|
ls -altr /dev/disk/by-path/*lun-4
|
|
|
|
|
|
TODO: is the `... --rescan` necessary?
|
|
|
|
|
|
3. find the associated WWID:
|
|
|
|
|
|
/lib/udev/scsi_id -g -u -d /dev/sdl
|
|
|
|
|
|
4. configure the disk on all Ganeti nodes, in Puppet's
|
|
|
2. configure the disk on all Ganeti nodes, in Puppet's
|
|
|
`profile::ganeti::chi` class:
|
|
|
|
|
|
iscsi::multipath::alias { 'web-chi-03':
|
|
|
wwid => '36782bcb00063c6a500000d67603f7abf',
|
|
|
}
|
|
|
|
|
|
5. confirm that multipath works, it should look something like this":
|
|
|
3. propagate the magic to all nodes in the cluster:
|
|
|
|
|
|
gnt-cluster command "puppet agent -t ; iscsiadm -m node --rescan ; multipath -r"
|
|
|
|
|
|
4. confirm that multipath works, it should look something like this":
|
|
|
|
|
|
root@chi-node-01:~# multipath -ll
|
|
|
web-chi-03-srv (36782bcb00063c6a500000d67603f7abf) dm-20 DELL,MD32xxi
|
... | ... | @@ -210,42 +210,31 @@ easily. The following is how `web-chi-03` was setup. |
|
|
|
|
|
and the device `/dev/mapper/web-chi-03` should exist.
|
|
|
|
|
|
5. partition the disk with `parted`:
|
|
|
|
|
|
parted --script --align optimal /dev/mapper/web-chi-03 \
|
|
|
mklabel gpt \
|
|
|
mkpart primary 0% 8MB \
|
|
|
set 1 bios_grub on \
|
|
|
mkpart primary 8MB 10008MB \
|
|
|
mkpart primary 10008MB 18008MB \
|
|
|
mkpart primary 18008MB 100%
|
|
|
|
|
|
TODO: this is one step that would be skipped if we have one iSCSI
|
|
|
volume per partition, obviously.
|
|
|
|
|
|
TODO: we probably do not need that `bios_grub` partition either.
|
|
|
|
|
|
6. adopt the disks in Ganeti:
|
|
|
|
|
|
gnt-instance add \
|
|
|
-n chi-node-04.torproject.org \
|
|
|
-o debootstrap+buster \
|
|
|
-t drbd --no-wait-for-sync \
|
|
|
-t blockdev --no-wait-for-sync \
|
|
|
--net 0:ip=pool,network=gnt-chi-01 \
|
|
|
--no-ip-check \
|
|
|
--no-name-check \
|
|
|
--disk 0:adopt=/dev/disk/by-id/dm-name-web-chi-03-part2 \
|
|
|
--disk 1:adopt=/dev/disk/by-id/dm-name-web-chi-03-part3,name=swap \
|
|
|
--disk 2:adopt=/dev/disk/by-id/dm-name-web-chi-03-part4
|
|
|
--backend-parameters memory=8g,vcpus=2 \
|
|
|
web-chi-03.torproject.org
|
|
|
--disk 0:adopt=/dev/disk/by-id/dm-name-tb-builder-03-root \
|
|
|
--disk 1:adopt=/dev/disk/by-id/dm-name-tb-builder-03-swap,name=swap \
|
|
|
--disk 2:adopt=/dev/disk/by-id/dm-name-tb-builder-03-srv \
|
|
|
--backend-parameters memory=16g,vcpus=8 \
|
|
|
tb-builder-03.torproject.org
|
|
|
|
|
|
NOTE: the actual node must be manually picked because the `hail`
|
|
|
allocator doesn't seem to know about block devices.
|
|
|
|
|
|
7. at this point, the VM probably doesn't boot, because for some
|
|
|
reason the `gnt-instance-debootstrap` doesn't fire when disks are
|
|
|
adopted. so you need to reinstall the machine, which involves
|
|
|
stopping it first:
|
|
|
|
|
|
gnt-instance shutdown --timeout=0 web-chi-03
|
|
|
gnt-instance reinstall
|
|
|
gnt-instance shutdown --timeout=0 tb-builder-03
|
|
|
gnt-instance reinstall tb-builder-03
|
|
|
|
|
|
From here on, follow the [next steps](#next-steps) above.
|
|
|
|
... | ... | |