Jonas Šukys

What if I told you that a single, tiny scalar parameter in a Large Language Model could determine whether it produces brilliant text—or complete gibberish? This is precisely what researchers at Apple have discovered in their fascinating paper on super weights (link: https://lnkd.in/dvTUiijY). In their study, they found that pruning a single scalar weight—what they call the super weight—in the Llama-7B model completely destroys its ability to generate coherent text. For example: • Before pruning: Llama-7B generates a “reasonable completion.” • After pruning: It produces “complete gibberish.” These super weights, while incredibly rare, are disproportionately critical to a model’s performance. Here’s another striking case study: • Prompt: “Summer is hot. Winter is “ • With the super weight: The model confidently predicts “cold” with an 81.4% probability. • Without the super weight: The model awkwardly guesses “the”, with a low probability of just 9.0%. What Makes Super Weights Special? 1. Stopword Suppression: Super weights ensure models focus on semantically meaningful words instead of common stopwords like “the” or “and.” 2. Super Activations: These weights induce rare, high-magnitude activations that propagate through the network, significantly influencing outputs. 3. Robust Compression: The authors propose a clever quantization technique: preserve super weights at full precision while quantizing the rest. This hardware-friendly, data-free method achieves results competitive with more complex quantization strategies. Why This Research Matters: • Efficiency: Understanding and preserving super weights enables effective compression of LLMs for deployment in resource-constrained environments. • Insight: It highlights that even the most massive models rely heavily on a few critical components. This work by Apple is a great reminder that sometimes, the smallest details make the biggest difference. You can read the full paper here: https://lnkd.in/dvTUiijY.

36

1 Comment

The paper "𝐂𝐚𝐧 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬 𝐔𝐧𝐥𝐨𝐜𝐤 𝐍𝐨𝐯𝐞𝐥 𝐒𝐜𝐢𝐞𝐧𝐭𝐢𝐟𝐢𝐜 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐈𝐝𝐞𝐚𝐬?" is making the rounds. On one hand, it supports a totally legitimate use of LLMs. LLMs for generating ideas. By mixing concepts plausibly, they're great at it. If you're scratching your head on something, it's always worth asking an LLM about it. At the least, it'll surface topics you just couldn't recall. On the other hand, some readers have egregiously extrapolated from this, suggesting LLMs can propose research ideas that are like to *work*. That's not at all what's shown here. All that is shown is that the LLM generates ideas that jive with a scientist's intuitions. There is a massive gulf between ideas that are plausible and ideas that work - that's the job of experimentation and it's what makes science hard. Very little can be said about lessening that gulf with this paper.

25

2 Comments

🔍📖 ***Key Insights: Empirical Study of "Catastrophic Forgetting" in LLMs!***🤖 The issue of "Catastrophic Interference" was originally coined by McCloskey and Cohen (1989), concluding that: 1) At least some interference will occur whenever new learning alters the weights involved in representation. 2) The greater the amount of new learning, the greater the disruption in old knowledge. 3) Interference is catastrophic in backpropagation networks when learning is sequential but not concurrent. Now, we commonly know this phenomenon as "Catastrophic Forgetting," where a model forgets previously learned information as it acquires new knowledge. I recently explored the insightful paper titled "An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning" published this April. 👉 https://lnkd.in/gNPZFgNU 📌 Key Contributions of the Study: Empirical Analysis of CF: An initial empirical study analyzing CF during continual instruction tuning, focusing on general knowledge areas such as domain knowledge, reasoning, reading comprehension, and bias. Evidence of CF Across Models: The research provides preliminary evidence that the CF problem is prevalent in the continual instruction tuning process across various models, including BLOOMZ, mT0, LLAMA, and ALPACA. The study highlights how model architecture, scale, and general instruction tuning influence the severity of the CF problem. 🔧 Effect of Model Architecture The findings show that decoder-only models can retain more knowledge during continual instruction tuning. Specifically, as model scale increases, decoder-only models like BLOOMZ exhibit less CF compared to encoder-decoder models like mT0. 🪂 Conclusion The findings reveal that CF is prevalent in the continual fine-tuning of various LLMs, with an increasing severity of forgetting in domain knowledge, reasoning abilities, and reading comprehension skills as model scale increases. The paper also suggests that employing general instruction tuning techniques may help alleviate the CF problem in LLMs, indicating that exploring effective methods to mitigate CF during continual fine-tuning is a promising research direction. #AI #CatastrophicForgetting #LanguageModels #ContinualLearning #LLMs #DataScience #ArtificialIntelligence #MachineLearning

1

⬇️ Replications are important! Here is a collection of replication studies in the field of Agricultural and Applied Economics in the journal Applied Economic Perspectives and Policy that Robert Finger, Arne Henningsen and I co-edited. https://lnkd.in/gAmnq-aH

24

2 Comments

It's called Computer *Science*... A thought-provoking paper on the arXiv from earlier this month. "Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations" https://lnkd.in/e6BNMhgZ Basically, Evan Miller at Anthropic is saying that we should treat evaluating LLMs the same as any other scientific experiment, i.e. with an appropriate level of statistical rigour (and provides recommendations). A few key points (about Q&A evals): • Judging performance of LLMs from metrics without any quantification of the error attached to those metrics is likely to be misleading. The authors suggest adding standard errors in parentheses after the (mean) metric as is standard in the sciences. • Central Limit Theorem applies for question scorings with finite variance and a large number of questions -> standard error/confidence intervals. • When questions are very similar, or related to the same source, they are no longer independent. This can make models look more performant on tasks than they are in general. Hence, clustered standard errors should be computed to adjust for the smaller effective number of questions. Not everyone is omitting error metrics on their evaluations, but take extra care when they are missing! #AITransformation #RaivenAI #LLM #AI

11

2 Comments

Recently I've been playing with "Swarm Intelligence", wondering in the back of my mind how I might apply it to my "day job" and merge with modern methods in the age of transformers...etc. Today I came across a paper (https://lnkd.in/g_bUdGB7) in the TL;DR newsletter that uses many inferences from small models to generate much better solutions than individuals, and at 3x lower cost than using a large model. And it is merely generating several samples independently without any interaction among the models. What if we instead applied a swarm intelligence algorithm to this approach so that inference N was informed by all the prior inferences? Would we better navigate the exploit/explore trade-off? This is the sort of place my video series will be heading. Relevant quote from the abstract: "...when it comes to inference, we often limit the amount of compute to only one attempt per problem. Here, we explore inference compute as another axis for scaling by increasing the number of generated samples. Across multiple tasks and models, we observe that coverage - the fraction of problems solved by any attempt - scales with the number of samples over four orders of magnitude. In domains like coding and formal proofs, where all answers can be automatically verified, these increases in coverage directly translate into improved performance. When we apply repeated sampling to SWE-bench Lite, the fraction of issues solved with DeepSeek-V2-Coder-Instruct increases from 15.9% with one sample to 56% with 250 samples, outperforming the single-attempt state-of-the-art of 43% which uses more capable frontier models."

50

3 Comments

The most brilliant data scientists I've encountered share a vital trait: 🔍 They're creative problem-solvers. It's easy to get hung up on the analytical bits of data science and miss the forest for the trees. But honestly, what's the point of being able to craft an algorithm from the ground up if you're stumped by the quirky, out-of-the-box challenges? Creativity isn't just a 'nice-to-have'; it's essential. And while some of it might be wired into us, there's plenty you can do to give your creative muscle a good workout: 📚 **Read**: Dive into a novel. Fiction has this magic of constructing entire universes in your mind, offering a slightly richer experience than audiobooks. 🎨 **Create**: Whether it's journaling, sketching, sewing, or colouring, engaging in any form of creative expression can do wonders. 🏃‍♀️ **Move**: Your body isn't built for endless hours at the desk. Cycle, walk, yoga, or skate your way to clarity and inspiration. 🗣 **Communicate**: Sometimes, a good chat is all you need to untangle a tricky problem. Do you think creativity is the backbone of data science? How do you keep your creative juices flowing? Share your thoughts below! 🧠✨

5

1 Comment

Check out Reuven M. Lerner presenting at #EuroPython2024, in their words: "In this talk, I introduce PyArrow, tell you what it does, how we can already use it in our Pandas work, and whether that’s a good idea." https://lnkd.in/d5vi8i_N 🐍

15

I somehow guessed (it was a wild guess :-) of course) that the two topics of my interest 1) Quantum Walks (3-years old interest) and 2) Simplicial Complexes and Homology (4-months old interest) could somehow be connected to derive new results. And I was expecting that those results would lead to something different and away from Topological Data Analysis (TDA). However, today I found a paper which does connect the two theories only to apply them back to TDA! :-) https://lnkd.in/g_6CQQeZ

18

7 Comments

This is really awesome. I used notebookLM to summarise a recent paper that shows how o1 like model can be reproduced. After providing the pdf from Arxiv, I noticed that the generated podcast was almost 49 minutes long. Very different from the earlier 10-12 minute version. And another impressive feature was the interactive mode where you can interject the two hosts and ask a question which they very graciously respond to. I wanted to share the audio summary but YT didn’t allow me to upload the WAV file. YT expects a video mp4 file. So I found free videos from Pexels and stitched them together with ffmpeg. The videos are meditative acquarium shots of fish swimming around. And with the audio discussing reproducing o1 as an engaging podcast episode, the video keeps the distraction to the minimum. The paper itself is a great read but if you have a 30-40 minute commute, or want to watch some cool fish swimming around and learn about o1, check out

12

Improving your RAG accuracy!! Paper: "Superposition Prompting: Improving and Accelerating Retrieval Augmented" I have been recently working on trying to improve the accuracy of a system trying to avoid "distraction phenomenon". Hypothesis was to make the context length smaller based on some smart chunking strategy and it has given good starting point to bring up a stable RAG for now. But this work will definitely help us or for anyone looking to improvise.

7

"Are these models understanding and reasoning in a general way, or are they doing what AI researcher Subbarao Kambhampati calls 'approximate retrieval' -relying on patterns of text that are contained in the model’s training data? The #CounterfactualTask paradigm can help answer this question. --> In this paradigm, models are evaluated on pairs of tasks that require the same types of abstraction and reasoning, but for each pair, the content of the first task is likely to be similar to training data, whereas the content of the second task (a “counterfactual task”) is designed to be unlikely to be similar to training data." An article that you should put in your reading list, by Prof. Melanie Mitchell. https://lnkd.in/dpXunq7b

1

Love this take (and this question) from Ravid Shwartz Ziv . My personal view is that current frontier AI is so far from the threshold of "Intern-AGI" that it would be difficult to produce a benchmark for it where results were greater than 0. But there are some subcomponents that such an agent would need, that we could probably benchmark on their own. * Complex long-chain instruction following with wrong and missing steps and adversarial perturbations * Formal self validation for task completion, rejection or redirection defining a policy with 99.9 trajectory match with specific groups of humans. IE, a SWE Intern should be able to guess, with 99.9 percent accuracy or better, whether it has completed a required subtask for its meta task, or needs more help * A zero shot understanding of how to navigate complex space layouts and estimate the time cost of actions with very very high precision

7

🔵 Week 3 : AI Paper by Hand ✍ --Paper 3 ➡️ RuAG: Learned-Rule-Augmented Generation For Large Language Models-- Combine RAG and Monte Carlo Tree Search (MCTS) and we get RuAG, a very interesting approach to instill data into an LLM. Here is a pdf of my annotated version of the paper, hope this helps you grab the central essence of this cool idea! 💡 🟢 A quick summary of what I understood: ----------------------------------------------------- RuAG (learned-rule-augmented generation) is a novel framework that distills large volume of information into logic rules using MCTS which are translated into natural language for seamless augmentation of an LLM's knowledge. --------- Central Idea ---------- The way RuAG works can be summarized as: 1. It starts with features (body predicate) and labels (target predicate). Target predicate is defined to be task-relevant while the body predicates are initialized as all the data attributions in the dataset. Based on task and description, LLM generates new target predicates and eliminates most irrelevant body predicates. 2. Next, RuAG uses an automated formulation for MCTS to generate set of first-order logic rules. It involves building a search tree and simulating outcomes to estimate the value of actions. To do so, it starts by build a search tree, selects the most promising child node, calculates the reward for the new node and propagates it up the tree. 3. Finally, the search rules are cleaned and translated into natural language which is then used for improved LLM generation. --------- Results ---------- - For Relation Extraction, RuAG achieves an F1-score of 60.42% - For Anomaly Detection, RuAG achieves an F1-score of 92.59%. In both cases, RuAG outperforms all baselines 💡 Paper : https://lnkd.in/g7BGrrv3 [Repost ♻️ ] Please help me share this resource to your network! #aibyhand #deeplearning #neuralnetworks #ai

84

1 Comment

Quality in, Garbage out. Briefly, this recent work on Nature ( among others ) claims and gives evidence that LLMs seriously degrade their behavior if trained from LLM generated data. To the point of wreaking havoc if the process is repeated 4-5 times. This says a lot about how lossy and noisy is LLM generated content wrt the human produced training data. Not a surprise. Nice read. "We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet."

64

9 Comments

LLMs are increasingly being used for evaluation, such as giving a score from 0 to 5 to a piece of text, based on certain criteria. Researchers found that using an additive scale works better than a direct mapping to a score, as it allows LLMs to reason about each additional point. Also, asking the LLMs to give it's reason before giving a score seems to be more effective. Since LLMs predict tokens sequentially, the reason will be used when generating the score, making it more consistent. Below is an example from Huggingface, which recently used this technique to filter for web pages with high educational content. Below is an extract from a web page. Evaluate whether the page has a high educational value and could be useful in an educational setting for teaching from primary school to grade school levels using the additive 5-point scoring system described below. Points are accumulated based on the satisfaction of each criterion: - Add 1 point if the extract provides some basic information relevant to educational topics, even if it includes some irrelevant or non-academic content like advertisements and promotional material. - Add another point if the extract addresses certain elements pertinent to education but does not align closely with educational standards. It might mix educational content with non-educational material, offering a superficial overview of potentially useful topics, or presenting information in a disorganized manner and incoherent writing style. - Award a third point if the extract is appropriate for educational use and introduces key concepts relevant to school curricula. It is coherent though it may not be comprehensive or could include some extraneous information. It may resemble an introductory section of a textbook or a basic tutorial that is suitable for learning but has notable limitations like treating concepts that are too complex for grade school students. - Grant a fourth point if the extract highly relevant and beneficial for educational purposes for a level not higher than grade school, exhibiting a clear and consistent writing style. It could be similar to a chapter from a textbook or a tutorial, offering substantial educational content, including exercises and solutions, with minimal irrelevant information, and the concepts aren't too advanced for grade school students. The content is coherent, focused, and valuable for structured learning. - Bestow a fifth point if the extract is outstanding in its educational value, perfectly suited for teaching either at primary school or grade school. It follows detailed reasoning, the writing style is easy to follow and offers profound and thorough insights into the subject matter, devoid of any non-educational or complex content. The extract: <EXAMPLE>. After examining the extract: - Briefly justify your total score, up to 100 words. - Conclude with the score using the format: "Educational score: <total points>" Source: https://lnkd.in/gnSXZPbP #llms #huggingface

21

5 Comments

The researchers behind the Kolmogorov-Arnold Network (#KAN) have questioned 🤔 some fundamental concepts underpinning MLPs, which are the basic elements of today's #ML and #AI solutions. They've done this by: 1-💡 Substituting the Universal Approximation Theorem with the Kolmogorov-Arnold Representation Theory. This substitution permits more flexible combinations of composition functions. 💪 2-💡 Exchanging the linear weights and fixed activation functions with activation functions and summation operations on the nodes and edges respectively. This exchange enhances performance for networks of comparable sizes. 🏋‍♀️ 3-💡 Offering interpretability through the use of l1 norm and entropy regularization parameters, followed by pruning. This approach enables clear graphical visualization with potential symbolification, establishing KAN as, potentially, #trustworthyai architecture. 🙌 This work underscores the importance of having a solid understanding of mathematics and functional data analysis. It urges revisiting those 'boring' math and data science theories and concepts. One particularly noteworthy resource is "Functional Data Analysis" by J.O. Ramsay and B.W. Silverman (2005). In this book, I encountered interesting concepts like b-Splines. Lui et al. utilized b-Splines in their initial KAN implementation. Various alternatives have been proposed, including Chebyshev polynomials, with more innovations expected. If you're interested in collaborating, don't hesitate to send a direct message. KAN paper: https://lnkd.in/e6UcqCTg KA representation theorem revised paper: https://lnkd.in/e6vEmiwK FDA book: https://lnkd.in/en7rB-3b

47

4 Comments

BERT (Bidirectional Encoder Representations from Transformers) If you're interested in understanding the inner workings of BERT (Bidirectional Encoder Representations from Transformers), check out this amazing research series by Chris McCormick. This 9 video series breaks down the foundational concepts behind BERT, including attention mechanisms, transformers, and the innovative WordPiece embeddings. Whether you're just getting started with NLP or diving deep into the details of state-of-the-art models, this series is an incredible resource. Start learning here: [https://lnkd.in/gDjpvCuC] Don't miss out on these clear and insightful explanations!

1