slash inference costs by doing a bunch of useless inferences in the hope that the one the user actually wanted happened to be one of them.
It cannot be more efficient than just waiting for the input and inferring once based on that.
this post was submitted on 23 Apr 2025
6 points (100.0% liked)
Artificial Intelligence
1596 readers
2 users here now
Welcome to the AI Community!
Let's explore AI passionately, foster innovation, and learn together. Follow these guidelines for a vibrant and respectful community:
- Be kind and respectful.
- Share high-quality contributions.
- Stay on-topic.
- Enhance accessibility.
- Verify information.
- Encourage meaningful discussions.
You can access the AI Wiki at the following link: AI Wiki
Let's create a thriving AI community together!
founded 2 years ago
MODERATORS
i mean…
your brain essentially does this it’s just that compute and memory are one system and it is as physically optimized as possible in brain systems.
this strategy is less stupid than it sounds if you abandon von neumann purism imo
it can be more efficient than just waiting for the input and inferring once based on that… you are an example of this in real life.
faster != more efficient. And you cannot compare brains to computers. Speculative execution improves speed in the cpu, at the cost of efficiency