Computing servers – AMD vs Intel?

Does anybody have any experience of computing performance of AMD vs Intel systems?

My general understanding of comparing the platforms e.g. Intel Xeon E5-2643v4 (3.40GHz 20MB) vs AMD Opteron 6348 (2.8GHz 16MB) is that the Intel is faster per core, better memory bandwidth, however AMD is cheaper to the extent that a 4xCPU AMD system costs less than a 2x CPU Intel CPU system.

I’m putting together a Linux server which is going to store a ~30TB database and have some computing power to do analytics on the database and run simulations. Don’t know exactly what software will be used yet. 🙄

Don’t know exactly what software will be used yet.

Then I would speculate that apart from CPU Spec or cpubenchmark.net comparisons, there’s nothing much you can do. Is that 30TB DB being stored on the same server and the same disks as the application layer? If so, then then in order to make a reasoned guess at performance then also look at disk subsystem performance too. If you can get twice as much AMD as you can Intel for the same money, you may want to consider the performance impact of separating the DB server from the application layer too.

I do wonder if you wouldn’t be better putting all of that on Amazon Cloud at least until you can do some profiling, if not permanently. Is local physical hardware really necessary?

And whether your app/requirements will take advantage of all those cores/threads.

Depending on where your server will be located, there’s also power consumption and generated heat (TDP) to consider. So with 4 AMD cpu’s there’ll be another comparative 190w to effectively cool.

If you’re building this yourself, what motherboard / ram are you planning on using?

The computational power requirements for fast disk access is much lower than your application server, I would always recommend separate physical boxes. If your SAN goes AWOL then it’s simply a case of deploying new storage, if your app server w/ storage calls it a day then you loose everything.

Also. a 30Tb DB and £1500 cpu’s, I would not consider launching a business app without redundant hardware in another DC / location that will ensure minimal business disruption.

Lastly, if you haven’t already, have a read of what Backblaze have to say about disks: https://www.backblaze.com/blog/hard-drive-reliability-stats-q1-2016/

Perhaps AWS / Azure / whatever could be a better solution – that redundancy is already built in, you need extra storage capacity or compute power, it’s very simple to expand without the requirements of carrying out a hardware upgrade to your machine(s) at some ungodly hour so there’s no disruption.

Thanks, yes I did identify it would be a good idea to have separate physical volumes for applications and database, essentially I’m considering SAS-SSD volume for Boot/applications and SAS-HDD volume for the DB all going though a card with a 1GB cache. I haven’t ever specced up stuff like this before but I am used to tinkering with hardware and hoping that I have some kind of clue what I am doing 🙂
The cpubenchmark.net scores indicate that AMD is much better value but of course that is a very synthetic test.
It all has to be on site for reasons beyond my control.
Oh, and I probably won’t be building it myself, I’m just trying to work out what specs to go for.

Woaaaaaah there!

New AMD Zen/Ryzen/Summit Ridge chips to launch very very soon, better take that into consideration.

Have a look at FreeNAS for your storage needs – as capable as Dell-EMC / et al and so long as you get the right hardware, super super super reliable and stable and of course, zero licensing costs.

1GB cache is pretty small. With FreeNAS you can specify a single/mirrored SSD to have 000’s of GB too.

Something like this chassis: https://www.supermicro.com/products/chassis/4U/846/SC846BE1C-R1K03JBOD calculate the details of the raid structure + qty of drives per ‘pool’ https://jsfiddle.net/Biduleohm/paq5u7z5/1/embedded/result/ and here’s some bedtime reading: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#RAIDZ_Configuration_Requirements_and_Recommendations

this is useful too: http://softwareengineering.stackexchange.com/questions/278802/calculating-disk-capacity-and-max-data-transfer-rate-of-a-hard-drive

You can also do point-to-point SAN/App server using something like 40Gb fiber too. Though with only 24 disks, you’d need to calculate whether you have enough throughput bandwidth.

I wouldn’t bother with SSD’s unless you’re going to use something like a 96x bay chassis and you’re going to populate it with >=1Tb disks and of course, 40Gb isn’t fast enough either.

I wouldn’t bother with SAS either – propriatory disks which cost more than SATA and very little differences in throughput.

A compute solution may be better as a cluster if it’s possible to make the compute tasks parallel. Can be cheaper per node and maybe similar price as a whole but potentially a lot more powerful.

You can even do this with Azure/Amazon to a degree, though the distributed nature of the comms and physical hardware could be an issue. I believe they do offer cloud solutions that are more suited for this though.

Or… a Pi cluster! Dead cheap per node. 😀

What the database software and has licensing being considered?

If you are using something you are paying for then less but faster CPUs might be worth considering, tin is cheap but software ‘can’ be expensive

but generally I’d rate memory and IO bandwidth over CPU for a database server.

I’m considering SAS-SSD volume for Boot/applications and SAS-HDD volume for the DB all going though a card with a 1GB cache.

Have you factored RAID into the equation?

Are you speccing an actual server, or a PC in a big box?

I’d be looking at something like this:

http://marketplace.hpe.com/pdp?catId=15351&reqCatId=7271241&catlevelmulti=15351_3328412_241475_7271241&prodNum=860806-425&country=UK&locale=en

As far as I’m concerned in the server world, there’s HP and there’s everything else.

You can get 10TB LFF disks now. Eight of those in RAID10 will be fastest and give you 40TB to play with; five in RAID5 will give you similar, but you’ll take a hit on write performance and it’ll take substantially longer to rebuild after a failed drive.

For the OS, you could use a separate smaller RAID1 pair, or if you’re not fussed about downtime should it fail you can install the OS to an internal USB pendrive / SD card (I don’t know offhand which one that server has, but there will almost certainly be one or the other).

You say it’s remote; if you’re tasked with supporting it you’ve also got iLO, which means that you can access it remotely if it dies on its arse.

AMD vs Intel, meh.

I can hook you up with a decent supplier if you like.

Cloud could be an option but its not a sunk cost, it is opex (which is an issue for some). Also depending on how long you are keeping it there is break even point. You also got to consider you internet pipe to the cloud you are using.

I wouldn’t bother with SAS either – propriatory disks which cost more than SATA

There’s nothing proprietary about SAS, it’s SCSI in fancy trousers (SATA is to IDE as SAS is to SCSI). On a single-use system you’re right, it’s pointless, but for a multi-user database server I’d go SAS all the way.

but generally I’d rate memory and IO bandwidth over CPU for a database server.

+1.

have some computing power to do analytics on the database and run simulations

If you’re even looking at Xeons and care about performance, go for the xeons.

AMD aren’t competitive just now on performance, though maybe are on cost in the mid/low range. Still, nearly everyone uses intel.

What sort of simulations, what sort of analytics? If you know this, surely you can answer the questions as to what sort of machine you need. If you don’t, ask whoever’s writing the sims/analysis code?

I do a lot of fairly high end computing, and the only thing that’d tempt me off Intel is power8 and I don’t have the budget for that…. that NVlink IO though…

(caveat to all the above, I do a lot of high end compute but DBs aren’t my thing at all)

30Tb DB is pretty big potatoes ime – make your life easier: draw up a clear breakdown of the demands you expect on the system, and start throwing that at a few suppliers for pricing and proposals.

Get a few of those proposals/quotes back (at least 2, but 3 or more would be ideal) and the hard work has been done for you. Then you can decide whether to simply buy one of the proposals, or build an equivalent system yourself. Also if it turns out the system doesn’t meet your needs, you can point at these proposals for backup, and it doesn’t look like you just pulled a spec out of thin air.

Nice to play with server specs and figure out what would work best for you, but at the end of the day, if the business is expecting this 30Tb DB to run nicely then I wouldn’t want to be the guy explaining why it runs like crap on the £20k of hardware the MD just signed off last month! 😀

30Tb DB is pretty big potatoes ime

Yeah. I’d have a couple of questions on the back of that.

1) Do you actually need that volume of data online all the time, or is it legacy crap going back a decade that no-one ever touches?

2) What’s the planned growth? If it’s 30TB now, how big is it going to be in ten years’ time? Generally speaking when speccing storage servers I’d take the required space and double it, which gets a bit eye-watering when you’re looking at 60TB.

jimwah speaks sense.

By courting some proposals, the actual system you require might come into better focus. You really do need to see and talk to the analytic team to understand how they expect to work.

Don’t discount stuff from proposals that offers to analyse the analysis – What people say they will do/use and how they actually use something is often quite different and only tends to diverge as time goes on.

With ‘ol fashioned OLAP & data warehousing the primary bottleneck was often the user – some lightweight coaching can massively reduce load and build good relations with the analytic staff.

Other than that, it’s I/O, ram and network. I’d say the bigger the data, er, dump the less the CPU gets stressed.

Edit: Oh, and try and separate data that is only read (never or infrequently updated) from data that is frequently written or updated.

Is it such big potatoes? My personal nas is 12TB (4x4TB raid 5) and cost only £200.

Currently there is 6TB of data, it is large because it involves lots of images. 30TB is an estimate on the size it will grow to, i was figuring on something like 8x8TB HDD raid 6=48TB. The needs are nearline, data neess to be there and searchable but for a small user pool, data required for simulation runs can be cached on the smaller SSD array if required.

My tender spec is very general, but i want to get my head around what works best too – otherwise how else am i going to assess the quality of what comes back.

I am one of the people who will be doing the actual work but the whole project is still in early stages. There will be another unit doing raytracing, which will certainly be xeons and GPU running windows. i am actually finding it quite hard work getting proper answers from the software supplier on what GPU is best for it! The general idea was to give the database server some extra ram/computing power and then it gives some flexibility to run linux/cpu orientated apps on it to take some load off the GPU/compute server.

Is it such big potatoes?

A single database of 30TB?

I reckon so; it’s not the enormous turnip, but it’s a sizeable number of potatoes.

i want to get my head around what works best too – otherwise how else am i going to assess the quality of what comes back.

Post them here for critique? (-: I said earlier I can recommend a supplier; this isn’t someone who’s going to sell you a box, but will help you to spec up what you need. She’s been doing it for me for years.

i was figuring on something like 8x8TB HDD raid 6=48TB

Bear in mind that as I said before, with several large drives in a RAID5 (or RAID6) configuration,

you’ll take a hit on write performance and it’ll take substantially longer to rebuild after a failed drive.

This may or may not be important to you over costs, I don’t know.

I’m kinda confused as to why you want a database server to be handling additional unrelated compute to take the load of dedicated GPU-heavy machines, incidentally. I don’t know your environment but that just feels wrong to me. Surely you’d be better off speccing the workstations properly in the first place.

One very critical bit of missing information here, is the IO requirement, amount of stuff is only one metric, 30TB is big but not massive, but you very much need a handle on how much of that stuff needs to be in play at any one time, how quickly, and what state of flux it’s going to be in.

A 30TB database of static data, well organised and indexed, being infrequently polled for a small subset of the data is a very different kettle of fish to even a 1TB database being frequently meddled with en-masse and in bulk, and will influence the storage and caching requiments massively.

A few big disks and some memory may be OK for the former, it will not necessarily for the latter…

No idea on the IT environment stuff, but what’s your continuity/backup requirements?

As you may need to purchase/setup (at least) twice the size of whatever you decide.

Also be careful with virtual infrastructure and large databases, you may find that you’ll dedicated hardware to get decent performance.

Also be careful with virtual infrastructure and large databases, you may find that you’ll dedicated hardware to get decent performance.

I dont think thats the case now, there is very little that I wouldn’t put on a hypervisor these days.

Back to the original simple question, why are you not looking at aws or azure?

Thanks for all the info.

The DB’s should mostly be pretty organised e.g. rows of data/keywords each associated with an image, but a lot of images.

I’ve got my own thoughts on what an ideal performance/budget system would be, basically some HGST He8’s for the main volume on RAID-Z2, with a pci-e SSD for a cache, OS/app layer on a RAID1 SSD volume, 2x 6 core 3.2Ghz Xeons and a large wedge of RAM. However, I’ve made my requirements even more general so I can see what different options vendors come up with.

The reason for considering computing on the DB server is is flexibility & trying to get the best on a limited budget. The cost of adding more RAM and better CPUs to the server allows some flexibility/options for sharing computing load and costs much less than an additional unit. There will be some analysis/computations done from the DB and maybe those can be done overnight when there are no users querying the DB for example.

Back to the original simple question, why are you not looking at aws or azure?

Because I’m not allowed to. I’m not able to outsource the specification either, although that was a good thought.

The DB’s should mostly be pretty organised e.g. rows of data/keywords each associated with an image, but a lot of images

So you want to put the image binary data in the DB?

As someone who works on the application side, I get nervous when people just talk about hardware….

Don’t know exactly what software will be used yet.

At that point, you ask your customer / colleague to come back when they do.

What’s the DB engine going to be?

But FWIW, I wouldn’t run any proper workloads on AMD servers, they may have been improved but always found they ran hot, ate power and the reason you got double the cores for Intel’s cost was because you needed them to equal it.

Avoid the AMD Opertons for Database Servers. Every Intel Xeon since 2011 has been a better investment – despite being able to uses lot’s of cores almost every database engine still benefits from fewer faster cores.

If your main apps are coded for Windows using .Net then SQL Server tends to be the DB Engine of choice. If you are considering this then the Intels make even more sense as it’s licensed per-core, as is Windows Server 2016. Less of an issue if Linux/PostgreSQL is where you are headed.

Given the workload descriptions given thus far, a relational database might not be best fit. But if you do go with SQL Server it offers FileTables and/or a Remote Blob Store which is great for use-cases where the storage of photos or documents is needed and keeps the files out of the database, but controlled by the database. Placing large binary files directly into a relational database is usually a recipe for disaster however hence, many NoSQL type solutions that can be a better chioce – depending on what you need to do.

With regards to size 30TB is big potatoes in database land. I work for a multi-national and we only have a couple of Data Warehouses bigger than that – but then we are not stuffing them full of jpegs…

Ah oops, yes true, it would be, maybe 300GB of db and a big blob of images which makes it an entirely different kettle of fish. The DB itself could perhaps be entirely cached in RAM.
In terms of the DB engine postgresql was mentioned a while back but I think some more discussion may be required (with people on holiday) to firm things up.
Thanks for all the info, I am on a steep learnding curve ATM.