I am sure inferencing can be done on new server cpu. They are dirt cheap compared to GPU. They might be bandwidth constrained but the raw storage can ease the load.
Artificial Intelligence thread
- Thread starter 9dashline
- Start date
you just need to start moving to Ascend NPUs for inference. It's a pain to switch your tech stack, but companies should be working on this.
Besides, they all have accumulated a whole bunch of AI chips already and smuggling will continue
Besides, they all have accumulated a whole bunch of AI chips already and smuggling will continue