Avinash Sharma V @avinv6

Intelligence in the brain is a heuristic way to handle NP-complete problems when exact deterministic solutions are computationally infeasible, and I do not know why intelligent agents are trying to replace deterministic workflows it makes no sense

Those who are bored this weekend, some stuff to explore - spintronics, magentic tunnel junction - thermodynamic computing, probablistic bit, @extropic we need breakthroughs in energy and computing!

We should have named it "simulated intelligence", cuz that's what it's doing. Simulating the output of intelligence

can gen ai and agents beat the productivity paradox / solow paradox?

Language is a representation of reasoned thought. Let me explain. No matter what the brain ends up doing to produce reasoning, once reasoning occurs, we use language to represent the 'output.' We use this representation to convey the reasoning to another person. It's safe to say that all well-reasoned thoughts can be represented as a stream of tokens in a language. This is the premise behind the reasoning models of today. They are not actually learning to reason, but instead, they are learning to generate a representation of reasoned thought.

I wonder if the role of really large models in the future will be to act as 'AI trainers' to distill knowledge into domain-specific, smaller reasoning models, which would be far more efficient in terms of compute and latency.

Reasoning models today work based on the idea that well-reasoned thoughts can be represented as a stream of tokens. Essentially, the goal is to train the model to generate specific token sequences that mimic reasoning processes. With Reinforcement Learning with Verifiable Rewards (RLVR), we take the posteriors from the model and use them as priors for the actions, training a reinforcement learning system accordingly. To maintain stability and prevent excessive policy drift, we incorporate a KL divergence control into the objective function.

@sama <100 years pass> 4. shot in the head!

At this point, the only meaning of life is to maximize the metric that represents the scientific and knowledge progress of humanity, ultimately reaching an inflection point where we are equipped to answer the question of "the true meaning of life." Unfortunately, right now, we don't have a better metric than the world's GDP.

@OpenAI , please tell me that - you have trained a network on the residual stream - and the "COT" equivalent in the newer o1 line of models is done before the unembedding.

Mechanistic interpretability and the auto encoders we train on the residual streams are awesome rabbit holes to go into!, I have my bets placed on the capability we are going to unlock to influence the network's "mood". It seems that if we just learn an auxilary network to adjust the neurons, we can intervene in the decision making process! @OpenAI , @AnthropicAI can we expect anything soon?

@deepigoyal I loved it! x.com/avinv6/status/…

Avinash Sharma V @avinv6

2 years ago

For those who dont understand the deal put forward by #zomato's founder @deepigoyal , here is some food for thought. For an inexperienced person who is looking to improve his/her prospects by doing an MBA from IIM or somewhere else, this deal is a gold mine. Of course the

0 0 0 90 0

For those who dont understand the deal put forward by #zomato's founder @deepigoyal , here is some food for thought. For an inexperienced person who is looking to improve his/her prospects by doing an MBA from IIM or somewhere else, this deal is a gold mine. Of course the founder will pick the smart ones from the pool, but what do they get? 1 year hands-on learning on how to run a unicorn company in india, work with industry experts, get to understand the segment and all this with an assurance of "free placement" from year 2, credibility and visibility built along the way, being part of the network of industry leaders and what not. Just pay the fee! I love to see someone come up with win-win deals once in a while breaking conventions with some out-of-the-box thinking! amazing deal @deepigoyal So guys, is your cup half empty?

Whoever is working on some sort of an AI hardware, guys, please we do not need just an improved hardware product. We need compute that can operate on 1/100th of the power consumed by today's compute. We do not need an "iphone for AI hardware" moment. We so badly need a "transistor for an AI" moment

@prajdabre1 NLL and KL divergence are equivalent while training. Even if its a streaming dataset, you are going to reuse the "seen" data repeatedly for your gradient based optimisations. With new data, new distribution is being modelled (currently) which is why we need "seen/past" data.

@Unspoken_Echoes 042

You of all people know that its not the specific state of weights in a neural network that matters but the fabric of learning mechanism which is mathematically designed to converge. Do you think the same is true with AI? the convergence mechanism being reckless capitalism and the AI arms race?. Looking at current AI and saying its not a problem is like looking at the weights at a particular instance of the network and saying the network is upto no good. I am not saying AI is an existential risk, but you ruling it out seems incorrect to me. If you are saying "now" is not the time to worry, you are probably right.

There might be an overlap between 2 and 3. The likelihood of intelligence and consciousness being an emergent behaviour from the mechanism of life/creation seems to be high. It is also possible that the entire question set you had in mind can be reduced to the first question presented.