From Words to Numbers.
Your large language model is secretly a capable regressor when given in-context examples.
We analyze how well pre-trained large language models (e.g., Llama2, GPT-4, Claude 3, etc) can do linear and non-linear #regression when given…
Microsoft presents Rho-1
Not All Tokens Are What You Need https://huggingface.co/papers/2404.
Previous language model pre-training methods have uniformly applied a next-token #prediction #LOSS to all training #tokens.
Continue reading “Rho-1: Not All Tokens Are What You Need” »
Want to experience totality again? Here are the next seven total solar eclipses across the world, from Alaska to Australia.
In “Star Trek: The Next Generation,” Captain Picard and the crew of the U.S.S. Enterprise leverage the Holodeck, an empty room capable of generating 3D environments, of preparing for missions and entertaining them, simulating everything from lush jungles to the London of Sherlock Holmes.
An enzyme in a cyanobacterium can take the unusual form a triangle containing ever-smaller triangular gaps, making a fractal pattern.
By Alex Wilkins
An international team of researchers led by groups from the Max Planck Institute in Marburg and Phillips University in Marburg has now discovered the first regular molecular fractal in nature. They discovered a microbial enzyme—citrate synthase from a cyanobacterium—that spontaneously assembles into a regular fractal pattern known as the Sierpiński triangle. This is an infinitely repeating series of triangles made up of smaller triangles.
“We stumbled on this structure completely by accident and almost couldn’t believe what we saw when we first took images of it using an electron microscope,” says first author Franziska Sendker.
“The protein makes these beautiful triangles and as the fractal grows, we see these larger and larger triangular voids in the middle of them, which is totally unlike any protein assembly we’ve ever seen before,” she continues.
Google announces Leave No Context Behind.
Efficient Infinite Context Transformers with Infini-attention.
This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and…
