Testing old 16 core Opteron 6386 SE on a Jingsha X89 motherboard

2019-03-09

For server market both AMD and Intel offer dedicated groups of products. Intel has Xeons processors, AMD had Opterons and now Epyc. One of AMD past family of server Opteron processors used a G34 socket and offered up to 16 cores and an option to have up to four CPUs on one motherboard for a total of 64 processor cores.

In this article I'll test the top of the line 16-core Opteron 6386 SE on a rather recently released Chinese G34 motherboard. How does 16-cores from 2012 compares to 12-16 core Threadrippers in 2019? Can it even beat latest quad core CPUs?

G34 Opterons

G34 socket was introduced in 2010 for 8-12 core Opterons from the Magny-Cours 6100 family made on a 45 nm node. In 2011 we got Interlagos 6200 processors made on 32 nm node and based on Buldozer architecture. This time AMD offered up to 16 cores per CPU. Last family to use this socket was Piledriver based 6300 Abu Dhabi family.

Opteron 6386 SE is the highest 16-core SKU that was available on the market with a release price of $1392. It offered 16 cores at 2.8 GHz base, 3.5 GHz half load turbo and 3.2 GHz full load turbo. You could get bit higher clocks on an eight core Opteron 6328 (3.2, 3.8 and 3.5 GHz respectively).

With the more cores meme it's worth to note that 6386 SE is not a true 16-core processor. It's a two 8-core processors in one package connected together with HyperTransport interconnect. Sort of a predecessor of Infinity Fabric and Zen design (more on that by the end of this article). Having two processors meant that they would run in a NUMA mode - each CPU has two own memory channels and accessing memory of other CPU is possible but at a performance penalty. I wrote more on NUMA in my dual Xeon PC overview. Going further - those 8-core CPUs are actually 8 integer cores and 4 floating point cores (units). Each two integer cores share one floating point core which makes the entire design bit complex and as we remember - Piledriver or Buldozer weren't the best processors on the market.

AMD had rather low IPC but could offer more cores than Intel as well as usually offer lower price and higher power consumption. For G34 you can put up to 4 processors on one motherboard for a total of 64 cores in a system. Opteron 6386 SE is rated at 140W TDP so you can imagine how much power a quad-CPU system would consume.

With proper OS support or even software optimization to handle load across multiple NUMA nodes such servers were a thing on the market and nowadays used parts can be still found on ebay or Chinese sites like Aliexpress or TaoBao. The processors seems to be quite common while motherboards tend to be much rare than for Xeons. Being rare gives speculative prices and you may see some G34 server boards more expensive than modern server and workstation boards. As offers tend to go and vanish even for current G34 Chinese boards I won't post much links here. You have to search for best prices for your location at given point of time on your own.

Jingsha X89 motherboard

G34 Opterons are discontinued and you won't find new hardware released for them... unless a Chinese company does that. After Huanan dual LGA2011 Xeon motherboard (a lot of reviews on YouTube if you are interested) it's time for Jingsha single socket G34 mATX motherboard.

This board shows up on some Aliexpress and ebay auctions under 80 USD on average, but the price can vary and sometimes goes on a sale or is listed for a bit absurd amount. Asus refurbished dual socket KGPE-D16 motherboards on Aliexpress are around 125 USD plus shipping.

Jingsha motherboard comes in consumer friendly micro-ATX form factor. We also get a custom CPU cooler, a SATA cable and a I/O shield. All the text on the box is in Chinese, no paper manual is provided.

As G34 Opterons use quad-channel memory configuration it's surprising to see only two slots for RAM - which then must be provided to the system in a odd-way - either as single channel per CPU on the MCM or two channels to one CPU and the other being left with indirect memory access over HT which impairs performance.

Opteron 6386 SE topology — Opteron 6386 SE on Jingsha board topology, looks like one node has all the RAM?

Aside of that we have four SATA II ports (not to be mistaken with SATA III which is the current version of the standard), one mSATA slot as well as x16 and x1 PCIe 2.0 slots. On the back we get 6 x USB 2, 2 x USB 3.0, Ethernet, mouse and keyboard PS2 ports and audio block. The motherboard supports up to 32GB of DDR3 RAM (1066/1333/1600 MHz registered ECC & non-EC modules).

As for power delivery on the board we have ISL6329 controller and six MDU1511 and MDU1514 mosfets which is likely a 5+1 phase design (ISL6329 can do up to 6-phase design with external PWM drivers). Mosfets are covered with a radiator (and thermal pad under it). Northbridge has a radiator and a fan where as southbridge has only a smaller radiator. The board uses a 24-pin ATX power connector alongside a 8-pin connector.

South and northbridge with power section in-between

Whole system drawing around 290W of power during some benchmarks

After installing the CPU, provided cooler, connecting an ATX power supply, dedicated graphics card (no integrated graphics on the processor) and powering it on the system booted without problems. The boot process was also quick. The board has a standard BIOS, no UEFI support (it's for quite old hardware after all). Windows 10 as well as Ubuntu 18.10 were used during the tests. Windows 10 shows this CPU as 8 core and 16 logical cores CPU. Some other apps just threat it as a 16-core part.

Provided radiator has a fan working at a fixed speed. It's not loud but still a calm fan noise can be heard. What is noticeable is the vibration it causes. When the motherboard wasn't screwed to the test bench the whole board could rattle from those vibrations. The small fan on northbridge thankfully doesn't spin fast enough to become audible and annoying.

The CPU-heatsink contact seems to be good

Noise level of the system — 43 dB on an open test bench with system on versus 24dB with system off

Noise frequency breakdown — Noise frequency breakedown

The mSATA port is next to the PCIe slot but shouldn't rather conflicth with a GPU. What I found problematic was unscrewing the locking screw.

Benchmarking 16 core Opteron 6386 SE

Most second-hand Opterons will be the mid-tier ones or even those with lowest clocks. With low IPC the Opteron systems had to offer multiple cores and good frequencies (which gets harder the more cores you offer) so if you want to create an Opteron system for something do check CPUs you are trying to buy. Trying to find Opteron 6386 SE may be hard as it's a rare part and that may create speculative pricing.

Also do note that the limit of two memory channels on the Jingsha motherboard can negatively affect the performance in memory intensive tasks and my benchmarks may not fully reflect the performance of this CPU when placed in a server motherboard.

The test system

CPU: Opteron 6386 SE
Motherboard: Jingsha X89
Power supply: Enermax revolution 87+ 850W
RAM: 2 x 8 GB Kingston KTH-PL313/8G ECC RDIMM
GPU: GTX 1050 Ti KalmX

Below are the results I got running various benchmarks:

Geekbench: single core: 2067, multi-core: 11533
V-Ray: 2:42 minutes
CPU-Z: multi-threaded 2349
Superposition: 5452 (1080p medium GTX 1050 Ti)
3DMark API overhead
3DMark Night Raid CPU Score: 2861
3DMark Time Spy CPU Score: 3900
openbenchmarking: Linux benchmark versus Ryzen and Threadripper
openbenchmarking: Linux system benchmarks
openbenchmarking: Linux multimedia benchmarks
openbenchmarking: Linux memory benchmarks
openbenchmarking: temperature and frequency monitoring during Kernel compilation

Phoronix Linux comparison benchmarks

C-Ray benchmark — C-Ray floating point CPU performance benchmark

Having limited amount of FPUs the 16-core Opteron 6386 SE looses to a quad core i7-7740X. The Opteron 6212 is a 8-core part of Interlagos generation and needs bit more than double the time of the 16-core Opteron.

Even if a benchmark is heavily parallel the 16-cores don't provide enough compute power to reach performance levels of modern, 5 or more years newer CPUs. Shows the generation leap AMD made with Zen architecture as well as Intel lead over AMD that lasted for many years.

When compared to some weaker consumer processors it looks bit better but still 6386 SE can't compete with old 2 x 6-core X5550 Xeons system in terms of timed Kernel compilation or even OpenArena FPS with GTX 1050 Ti on both.

CPU core 0 frequency during three Kernel compilations

CPU core 0 temperature during three Kernel compilations

CPU core 0 frequency during three C-ray runs

This CPU offers 2.8 GHz base, 3.5 GHz half load turbo and 3.2 GHz all core turbo. Clock boost seems to be working but the constantly jumping frequency and capped temperature reading may indicate a sort of thermal throttling with provided cooler. The test was made on an open test bench.

Windows benchmarks

Starcraft 2 for mass battles requires a lot of CPU performance in a pretty much single core scenario. In such the Opteron barely handles the required load.

The system works, but the performance is kind of disappointing if you are expecting anything close to modern more-than-four-core CPUs. Where single core performance or IPC in general will be essential the 16-core Opteron will loose to a quad core. In many benchmarks capable of scaling to both CPUs on a NUMA system the performance isn't that appealing. You could try a 4-socket video rendering PC for the fun of it but some people already tried to do something with those Opterons and the results were mixed - from very bad to mediocre. Depending on price it could be worth the fun but still there are tasks like high performance gaming that won't happen on this CPU.

Even though benchmarks may look bad and we could start making AMD memes again let's look at some of the technology in this system instead. Those Opterons used various means of communications - socket to socket, CPUs inside one MCM, RAM access etc. All of that was (in general) under the HyperTransport name and brand. It was used in AMD systems, but also by IBM, Apple or Nvidia nForce motherboard chipsets. HyperTransport was used in this case as an interconnect for NUMA multiprocessor systems. AMD used it with a proprietary cache coherency extension as part of their Direct Connect Architecture in their Opteron and Athlon 64 FX processor families. You can read more on Wikipedia.

I mentioned HyperTransport because nowadays AMD has Infinity Fabric in Zen based processors. This systems consists of Infinity Scalable Data Fabric (SDF) for data and Infinity Scalable Control Fabric (SCF) for system control signals. The SCF is a superset (extended version of) HyperTransport. You can read more on wikichip if you are interested in this technology. Even when the old Opteron may be dead in 2019 it still marks a one of technology points in history from which Zen architecture was born. New GPU or CPU designs takes years of iterations and technology advancement to materialize.

As for the Jingsha X89 board. Is it worth it? At a low enough price maybe, depending if you have a good used for the system - the one thing that is worrying for me is only two DIMM slots for what supposed to be a quad-channel system. If you want a good workstation - check for Threadripper sales or future rumored 12-16 core Ryzen CPUs or more modern Intel Xeon dual socket boards at worse.

Comment article