Price-to-Performance: Nvidia RTX 4000s vs AMD Radeon 7000s vs Intel ARC Discussion

[tusker]

This is fairly common question in the threads; but looking forward with all these GPU launches, where do you think the price to performance sweetspot is going to lie amongst the competitiors?

NVidia's RTX 4090 looks like the current king with ~16400 CUDA cores, but will DFL also leverage the massive uplift in Tensor Cores? The current pricing and additional hardware requirements also puts the 4000 Series far from reach for a lot of us currently.

AMD's RX 7000 Series is speculated to come with its own variation of Tensor cores and Performance uplifts. Traditionally AMD GPUs were priced competitively but fell behind on DFL performance. Perhaps the DFL will leverage the newer chips better.

Please, Log in or Register to view URLs content!

Intel's ARC GPUs are priced well currently and have good ML performance in apps like Topaz Video Enhance with their MXM cores. Plus the 16GB of VRAM at 350$ is superb. Has anyone tried DFL with these new chips?

 

[TMBV]

DFL was never able to utilize tensor cores due to low precision of the math they can do (fp16), not precise enough for DFL, there was an fp16 option added to DFL that would use them, it was added twice, removed twice and never worked (models would collapse, training not stable), not likely it will do now unless they offer higher precision of iperov rewrites how DFL talks to the gpus, perhaps still not full tensor core utilization but more low level access for better performance and memory management (less vram overhead and usage, thus higher model parameters possible).

Whatever AMD will have might have the same limiations as Nvidia's tensor cores implementation so good chance it will not be possible to utilize those extra cores for DFL.

As far as Intel goes, it should run on DX12 build like AMD, if it suprasses AMD in DX12 performance then it migh also be faster in DFL.

Currently sweetspot is 3060 with 12GB when it comes to price to performance for DFL, then you've got garbage gpus that are faster but have less VRAM so not that capable and then you get 3090 and Ti with 24GB. If Nvidia releases a 4060 that has 12GB but manages to be cheaper it will be the new sweetspot but we are all hoping for something slightly more expensive (3070/3080 price) with 16GB of VRAM.

 

[honor]

that's a good question ( and me specially like the way you asked that question )

 

[tusker]

Never knew DFL didn't work well with Tensor till now. Its unfortunate that the part of the GPU marketed towards ML and AI workloads doesn't gel with DFL. But finger crossed for a future implementation to work through.

Also from the way I read it, is it true that DFL was developed with NVidia GPUs or CUDA-specific workloads in mind? AMDs chips were quite close last generation for Rasterized Gaming Performance with NVidia.

On top of that, I doubt that NVidia will release a 4070 class GPU with 16GB anytime soon, not when the 4080 12 GB (What should've been the 4070) exists. Current SKU lineup and pricing for the 40 Series is bonkers enough.

Thanks for the quick reply though!

 

[TMBV]

Please, Log in or Register to view quote content!

It's not the case of the GPUs not getting along with DFL but DFL not getting along with the GPUs due to how it's written, the only person that can change it is iperov, perhaps that's what he is working on now with "DFL 3.0" (which may or may not exist or even become a thing).

 

[tusker]

Please, Log in or Register to view quote content!

I see. Till that's here, if it comes, I'll just stick to my 1080 for the current moment.
Though I really want to see what AMD has in stock next month before I commit to a 3070 / 3080.

 

[TMBV]

Please, Log in or Register to view quote content!

If you don't have 3000 series then don't buy it, go straight for whatever 4000 series with 12GB or 16GB (still nice upgrade from 8GB 1080 and speed increase alone will be worth it).
You could also wait for cheaper used 3090s but I doubt people will be selling many of them, we are still dealing with the post-shortage effects, many people didn't have those gpus for a while and many will keep them since they probably paid quite a lot for them, way more than msrp, if anyone will sell them it won't be for cheap, certainly there will be those hoping to sell these used gpus for msrp (good luck to them, only idiots would buy used gpus for msrp) just like it was when 3000 series came out and people were selling 2080Tis (except 3090 are really good gpus still and 2000 series was garbage because both rt corese and tensor cores were not properly developed, dlss didn't work well, rtx was too taxing).

If A770 does actually sell that cheap and has better performance than AMD then it might be a decent alternative to RTX3060 but that's only assuming it manages to be fast enough to actually be anywhere close to 3060, the thing is that the AMD specific DX12 build is nowhere near as fast as regular build, in addition there is no guarantee an Intel GPU will work on it and also there is no guarantee DX12 builds will work in the future, DFL's support for AMD is not the best (still better than Linux based on iperovs famous DFL commit comment "fuck linux").
Also while Topaz and other commercial stuff will work on Intel, many open source projects may not as it depends on the developers to support other gpus, until Intel has at least 10-15% market share in terms of dedicated gpus it will be ignored, stuff like ai frame interpolation, upscaling/enhancing, stable diffusion implementations and other similar stuff, all of it is almost always made for Nvidia first, then some get AMD support, Intel is at the dead end.

EDIT, it's bad:



Performance close to 3060, this means on DX12 build it will be probably much slower, not sure how but I bet at least 20-30% slower, assuming it works.
Bad drivers also don't sound like something one would want to have to deal with when you want stable operation 24/7.

 

[666VR999]

Does Iperov update DFL to make better use of newer versions of Cuda? I saw a pugetbench blog post on the 4090 for ML/AI and they said it was really great, and will get better with Cuda 12 that's coming out in maybe next 6 months.

Please, Log in or Register to view URLs content!

"Conclusions
These preliminary compute results for the new NVIDIA RTX 4090 look very good and offer a significant performance boost over last generation RTX 3090 (which was already very good)! I expect results to be better with code compiled against CUDA 12 which will have full support for Ada Lovelace and Hopper arch GPUs. There will be more RTX 4000 series and Pro Axxx-ada series GPUs over the next few months. I will revisit testing.

One surprising result was how respectable the double precision floating point (fp64) was on RTX 4090. fp64 performance is not a highlighted feature of RTX GPUs. Single precision fp32 is typically 20 times fp64 for these GPUs. However, the fp64 performance of the RTX 4090 is competitive with 16-34 core CPUs. I feel this could be used for code testing and development that is target to run on high-end compute GPUs like A100 and H100.

Overall it looks like NVIDIA has kept the performance increase near doubling on successive GPU generations for another release. They have been doing this for over 10 years now. Impressive indeed!"

 

[tusker]

Please, Log in or Register to view quote content!

Yeah, looks like ARC only stacks with 3060 with terrible compatibility issues. DFL looks like it'll remain on Nvidia's court for a while.
On the bright side, looks like NVidia scrapped 4080 12GB or rather claims its not named right. We might just see it come back as a 4070 or 4070Ti with slightly reduced prices.
If we can get a decent 4070 card 12GBish around the ~700$ mark, that will be brilliant.

Please, Log in or Register to view URLs content!

 

[TMBV]

Please, Log in or Register to view quote content!

700$ for 12GB card is awful deal, you can get 3060 for half that and it also has 12GB.

Please, Log in or Register to view URLs content!

 

[tusker]

Please, Log in or Register to view quote content!

True, but considering the markup they're selling the 40 series, and the expected 3090ish performance that the 4080 12GB was supposed to match I'd reckon 700$ is a realistic expectation looking at NVidia's current pricing.
I'd love to get it at 500 Though.

 

[TMBV]

Please, Log in or Register to view quote content!

I'd rather have 3080 level of performance with 16 or 24GB over 12GB with 3090 performance, waste of money.

 

[666VR999]

Hopefully I can find a 4090 in stock somewhere and finally join in!