Lifeboat Foundation

Global Workspace Theory (GWT)Information Integration Theory (IIT)

Daniel Dennett thinks that the Mind-Body problem has a solution. And moreover it’s not a specific or \.

Paper on the hubble tension.

Wendy L. Freedman and Barry F. Madore JCAP11(2023)050 DOI 10.1088÷1475−7516÷2023÷11÷050

Researchers at European XFEL in Schenefeld near Hamburg have taken a closer look at the formation of the first crystallization of nuclei in supercooled liquids. They found that the formation starts much later than previously assumed. The findings could help to better understand the creation of ice in clouds in the future and to describe some processes inside the Earth more precisely.

Reducing carbon emissions from small-scale combustion systems, such as boilers and other industrial equipment, is a key step towards building a more sustainable, carbon-neutral future. Boilers are widely used across various industries for essential processes like heating, steam generation, and power production, making them significant contributors to greenhouse gas emissions.

Grokked Transformers are Implicit Reasoners.

A mechanistic journey to the edge of generalization.

We study whether transformers can learn to implicitly reason over parametric knowledge, a skill that even the most capable language models struggle with.

Continue reading “Paper page — Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization” »

Light is renowned for its incredible speed.

This brand has presented the first water engine in history: 2,500 ºC and the end of hydrogen (and Tesla, of course)

Language models (LMs) are a cornerstone of artificial intelligence research, focusing on the ability to understand and generate human language. Researchers aim to enhance these models to perform various complex tasks, including natural language processing, translation, and creative writing. This field examines how LMs learn, adapt, and scale their capabilities with increasing computational resources. Understanding these scaling behaviors is essential for predicting future capabilities and optimizing the resources required for training and deploying these models.

The primary challenge in language model research is understanding how model performance scales with the amount of computational power and data used during training. This scaling is crucial for predicting future capabilities and optimizing resource use. Traditional methods require extensive training across multiple scales, which is computationally expensive and time-consuming. This creates a significant barrier for many researchers and engineers who need to understand these relationships to improve model development and application.

Existing research includes various frameworks and models for understanding language model performance. Notable among these are compute scaling laws, which analyze the relationship between computational resources and model capabilities. Tools like the Open LLM Leaderboard, LM Eval Harness, and benchmarks like MMLU, ARC-C, and HellaSwag are commonly used. Moreover, models such as LLaMA, GPT-Neo, and BLOOM provide diverse examples of how scaling laws can be practiced. These frameworks and benchmarks help researchers evaluate and optimize language model performance across different computational scales and tasks.