Flash Forward is a show about possible (and not so possible) future scenarios. What would the warranty on a sex robot look like? How would diplomacy work if we couldn’t lie? Could there ever be a fecal transplant black market? (Complicated, it wouldn’t, and yes, respectively, in case you’re curious.) Hosted and produced by award winning science journalist Rose Eveleth, each episode combines audio drama and journalism to go deep on potential tomorrows, and uncovers what those futures might re ...
…
continue reading
תוכן מסופק על ידי Igor Melnyk. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Igor Melnyk או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.
דומה לArxiv Papers
Artificial Intelligence has suddenly gone from the fringes of science to being everywhere. So how did we get here? And where's this all heading? In this new series of Science Friction, we're finding out.
…
continue reading
DNA science. Artificial intelligence. Smartphones and 3D printers. Science and technology have transformed the world we live in. But how did we get here? It wasn’t by accident. Well, sometimes it was. It was also the result of hard work, teamwork, and competition. And incredibly surprising moments. Hosted by bestselling author Steven Johnson (“How We Got To Now”), American Innovations uses immersive scenes to tell the stories of the scientists, engineers, and ordinary people behind the great ...
…
continue reading
Come dive into one of the curiously delightful conversations overheard at National Geographic’s headquarters, as we follow explorers, photographers, and scientists to the edges of our big, weird, beautiful world. Hosted by Peter Gwin and Amy Briggs.
…
continue reading
Interviews with Anthropologists about their New Books Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/anthropology
…
continue reading
Whether we wear a lab coat or haven't seen a test tube since grade school, science is shaping all of our lives. And that means we all have science stories to tell. Every year, we host dozens of live shows all over the country, featuring all kinds of storytellers - researchers, doctors, and engineers of course, but also patients, poets, comedians, cops, and more. Some of our stories are heartbreaking, others are hilarious, but they're all true and all very personal. Welcome to The Story Collider!
…
continue reading
Hannah Fry and Dara Ó Briain tackle listeners' conundrums with the power of science!
…
continue reading
The Science Show gives Australians unique insights into the latest scientific research and debate, from the physics of cricket to prime ministerial biorhythms.
…
continue reading
We humans could have a bright future ahead of us that lasts billions of years. But we have to survive the next 200 years first. Join Josh Clark of Stuff You Should Know for a 10-episode deep dive that explores the future of humanity and finds dangers we have never encountered before lurking just ahead. And if we humans are alone in the universe, if we don't survive intelligent life dies out with us too.
…
continue reading
Join Dr Karl Kruszelnicki, Lucy Smith and their scientific guests, with a bunch of curious triple j listeners for a weekly injection of science, myth-bashing and answers!
…
continue reading
Player FM - אפליקציית פודקאסט
התחל במצב לא מקוון עם האפליקציה Player FM !
התחל במצב לא מקוון עם האפליקציה Player FM !
))
[QA] Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs
Manage episode 445761057 series 3524393
תוכן מסופק על ידי Igor Melnyk. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Igor Melnyk או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.
The paper investigates extreme-token phenomena in transformer-based LLMs, revealing mechanisms behind attention sinks and proposing strategies to mitigate their impact during pretraining.
https://arxiv.org/abs//2410.13835
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1597 פרקים
שתפו
Manage episode 445761057 series 3524393
תוכן מסופק על ידי Igor Melnyk. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Igor Melnyk או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.
The paper investigates extreme-token phenomena in transformer-based LLMs, revealing mechanisms behind attention sinks and proposing strategies to mitigate their impact during pretraining.
https://arxiv.org/abs//2410.13835
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1597 פרקים
All episodes
×
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.
דומה לArxiv Papers
Flash Forward is a show about possible (and not so possible) future scenarios. What would the warranty on a sex robot look like? How would diplomacy work if we couldn’t lie? Could there ever be a fecal transplant black market? (Complicated, it wouldn’t, and yes, respectively, in case you’re curious.) Hosted and produced by award winning science journalist Rose Eveleth, each episode combines audio drama and journalism to go deep on potential tomorrows, and uncovers what those futures might re ...
…
continue reading
Artificial Intelligence has suddenly gone from the fringes of science to being everywhere. So how did we get here? And where's this all heading? In this new series of Science Friction, we're finding out.
…
continue reading
DNA science. Artificial intelligence. Smartphones and 3D printers. Science and technology have transformed the world we live in. But how did we get here? It wasn’t by accident. Well, sometimes it was. It was also the result of hard work, teamwork, and competition. And incredibly surprising moments. Hosted by bestselling author Steven Johnson (“How We Got To Now”), American Innovations uses immersive scenes to tell the stories of the scientists, engineers, and ordinary people behind the great ...
…
continue reading
Come dive into one of the curiously delightful conversations overheard at National Geographic’s headquarters, as we follow explorers, photographers, and scientists to the edges of our big, weird, beautiful world. Hosted by Peter Gwin and Amy Briggs.
…
continue reading
Interviews with Anthropologists about their New Books Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/anthropology
…
continue reading
Whether we wear a lab coat or haven't seen a test tube since grade school, science is shaping all of our lives. And that means we all have science stories to tell. Every year, we host dozens of live shows all over the country, featuring all kinds of storytellers - researchers, doctors, and engineers of course, but also patients, poets, comedians, cops, and more. Some of our stories are heartbreaking, others are hilarious, but they're all true and all very personal. Welcome to The Story Collider!
…
continue reading
Hannah Fry and Dara Ó Briain tackle listeners' conundrums with the power of science!
…
continue reading
The Science Show gives Australians unique insights into the latest scientific research and debate, from the physics of cricket to prime ministerial biorhythms.
…
continue reading
We humans could have a bright future ahead of us that lasts billions of years. But we have to survive the next 200 years first. Join Josh Clark of Stuff You Should Know for a 10-episode deep dive that explores the future of humanity and finds dangers we have never encountered before lurking just ahead. And if we humans are alone in the universe, if we don't survive intelligent life dies out with us too.
…
continue reading
Join Dr Karl Kruszelnicki, Lucy Smith and their scientific guests, with a bunch of curious triple j listeners for a weekly injection of science, myth-bashing and answers!
…
continue reading
Player FM - אפליקציית פודקאסט
התחל במצב לא מקוון עם האפליקציה Player FM !
התחל במצב לא מקוון עם האפליקציה Player FM !
