Skip to content
Snippets Groups Projects
Verified Commit 21fd8ea4 authored by anarcat's avatar anarcat
Browse files

add hardware details including photos from Quintex (team#40969)

parent babadee8
No related branches found
No related tags found
No related merge requests found
......@@ -320,24 +320,67 @@ terminalserver](https://grml.org/terminalserver/).
## Design and architecture
<!-- how this is built -->
<!-- should reuse and expand on the "proposed solution" discussed in -->
<!-- a previous RFC or the Discussion section below. it's a -->
<!-- "as-built" documented, whereas the "Proposed solution" is an -->
<!-- "architectural" document, which the final result might differ -->
<!-- from, sometimes significantly -->
The Quintex PoP is at the [Infomart](https://en.wikipedia.org/wiki/Infomart), a gigantic datacenter in
Dallas, Texas. We have our own switch there donated by Quintex, a
[D-Link DGS-1250-52X](https://www.dlink.com/en/products/dgs-1250-52x-52-port-10-gigabit-smart-managed-switch) switch. The servers are connected through the
different VLANs on that switch. The OOB management network is on a
separate "dumb" switch.
## Network topology
### Network topology
This is the *planned* network topology, not implemented yet.
This is the *planned* network topology, not fully implemented yet.
![network topology graph](quintex/topology.png)
TODO: document textually the above, see the cymru ref for an example
It physically looks like this:
![Photo of the top of a 42U cabinet with three servers and a switch](quintex/gnt-dal-2023-02-10.jpg)
![Details of the switch and servers, each server has 10 disk trays in front](quintex/gnt-dal-top-2023-02-10.jpg)
![Back of the setup, where we see the extra management switch](quintex/gnt-dal-top-2023-02-10.jpg)
## Services
<!-- open ports, daemons, cron jobs -->
The main service at this point of presence is a 3-machine Ganeti
cluster called `gnt-dal`.
### gnt-dal Hardware
Each machine is identical:
* [SuperMicro 1114CS-TNR 1U][]
* AMD Milan (EPYC) 7713P 64C/128T @ 2.00Ghz 256M cache
* 512G DDR4 RAM (8x64G)
* 2x Micron 7450 PRO, 480GB PCIe 4.0 NVMe*, M.2 SSD
* 6x Intel S4510 1.92T SATA3 SSD
* 2x Intel DC P4610 1.60T NVMe SSD
* Subtotal: **12,950$USD**
* Spares:
* Micron 7450 PRO, 480GB PCIe 4.0 NVMe*, M.2 SSD: 135$
* Intel® S4510, 1.92TB, 6Gb/s 2.5" SATA3 SSD(TLC), 1DWPD: 345$
* Intel® P4610, 1.6TB NVMe* 2.5" SSD(TLC), 3DWPD: 455$
* DIMM (64GB): 275$
* labour: 55$/server
* **Total**: **40,225$USD**
* TODO: final cost to be confirmed
* **Extras**, still missing:
* shipping costs: was around 250$ by this [shipping estimate][],
provider is charging 350$
* **Grand total**: **41,000$USD** (estimate)
[SuperMicro 1114CS-TNR 1U]: https://www.supermicro.com/en/Aplus/system/1U/1114/AS-1114CS-TNR.cfm
For three such servers, we have:
* 192 cores, 384 threads
* 1536GB RAM (1.5TB)
* 34.56TB SSD storage (17TB after RAID-1)
* 9.6TB NVMe storage (4.8TB after RAID-1)
* Total: **40,936$USD**
See [TPA-RFC-43](policy/tpa-rfc-43-cymru-migration-plan) for a more in-depth discussion of the chosen
hardware and location.
## Storage
......
howto/quintex/gnt-dal-2023-02-10.jpeg

760 KiB

howto/quintex/gnt-dal-back-2023-02-10.jpeg

706 KiB

howto/quintex/gnt-dal-top-2023-02-10.jpeg

815 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment