CASA: Cross-Attention via Self-Attention for Efficient Vision-Language Fusion
Paper
β’
2512.19535
β’
Published
β’
7
AI & ML interests
None defined yet.
Recent Activity
View all activity
nielsrΒ
submitted
a
paper
to Daily Papers
2 days ago
Post
1690
π¨ New tool for the
Nymbo/Tools MCP server: The new
How it works: The tool exposes the standard discover/info/resources/validate actions. Skills live in
I've included a
Caveat: On HF Spaces,
Try it out ~ https://www.nymbo.net/nymbot
Agent_Skills tool provides full support for Agent Skills (Claude Skills but open-source).How it works: The tool exposes the standard discover/info/resources/validate actions. Skills live in
/Skills under the same File_System root, and any bundled scripts run through Shell_Command, no new infrastructure required.Agent_Skills(action="discover") # List all available skills
Agent_Skills(action="info", skill_name="music-downloader") # Full SKILL.md
Agent_Skills(action="resources", skill_name="music-downloader") # Scripts, refs, assetsI've included a
music-downloader skill as a working demo, it wraps yt-dlp for YouTube/SoundCloud audio extraction.Caveat: On HF Spaces,
Shell_Command works for most tasks, but some operations (like YouTube downloads) are restricted due to the container environment. For full functionality, run the server locally on your machine.Try it out ~ https://www.nymbo.net/nymbot
Post
2601
Nvidia is on a roll lately. Nemotron 3 Nano is my new fav local model, but here's the real flex: they published the entire evaluation setup. Configs, prompts, logs, all of it. This is how you do open models π₯
https://huggingface.co/blog/nvidia/nemotron-3-nano-evaluation-recipe
https://huggingface.co/blog/nvidia/nemotron-3-nano-evaluation-recipe
Post
2292
Muon vs MuonClip vs Muon+Adamw
Muon has gone from an experiment to a mainstream optimizer, but does it hold up for fineβtuning? We ran headβtoβhead tests on Qwen3β4B (10k+ highβquality instruction rows) to find out.
Short story: Pure Muon converged fastest at the start, but its gradientβnorm spikes made training unstable. MuonClip (Kimi K2βs clipping) stabilizes long pretraining runs, yet in our smallβscale fineβtune it underperformed, lower token accuracy and slower convergence. The winner was the hybrid: Muon for 2D layers + AdamW for 1D layers. It delivered the best balance of stability and final performance and even beat vanilla AdamW.
Takeaway: for small-scale fine-tuning, hybrid = practical and reliable.
Next Step: scale to larger models/datasets to see if Muonβs spikes become catastrophic or if clipping wins out.
Full Blog Link: https://huggingface.co/blog/KingNish/optimizer-part1
Muon has gone from an experiment to a mainstream optimizer, but does it hold up for fineβtuning? We ran headβtoβhead tests on Qwen3β4B (10k+ highβquality instruction rows) to find out.
Short story: Pure Muon converged fastest at the start, but its gradientβnorm spikes made training unstable. MuonClip (Kimi K2βs clipping) stabilizes long pretraining runs, yet in our smallβscale fineβtune it underperformed, lower token accuracy and slower convergence. The winner was the hybrid: Muon for 2D layers + AdamW for 1D layers. It delivered the best balance of stability and final performance and even beat vanilla AdamW.
Takeaway: for small-scale fine-tuning, hybrid = practical and reliable.
Next Step: scale to larger models/datasets to see if Muonβs spikes become catastrophic or if clipping wins out.
Full Blog Link: https://huggingface.co/blog/KingNish/optimizer-part1
Post
2436
I tested Muon vs MuonClip vs Muon+AdamW for fine-tuning LLMs
Just published a blog on that, Read here π https://huggingface.co/blog/KingNish/optimizer-part1
Just published a blog on that, Read here π https://huggingface.co/blog/KingNish/optimizer-part1
- 1 reply
mrfakenameΒ
posted
an
update
21 days ago
Post
3871
Excited to share that I've joined the Hugging Face Fellows program! π€
Looking forward to contributing to & working more closely with the open-source ecosystem - huge thanks to everyone who's supported me on this journey! π
Looking forward to contributing to & working more closely with the open-source ecosystem - huge thanks to everyone who's supported me on this journey! π
christopherΒ
authored
a
paper
21 days ago
Post
4987
π I've just shipped a major update to the
Nymbo/Tools MCP server: the
Anthropic found 98.7% context savings using code execution with MCP, Cloudflare published similar findings. This is my open-source implementation of the same idea.
# The Problem
Traditional MCP exposes every tool definition directly to the model. With 12 tools, that's thousands of tokens consumed *before the conversation even starts*. Each tool call also passes intermediate results through the context window β a 10,000-row spreadsheet? That's all going into context just to sum a column.
# The Solution: One Tool to Rule Them All
Instead of the model making individual tool calls, it writes Python code that orchestrates the tools directly:
Don't know what tools are available? The agent can discover them at runtime:
The individual direct tool calls are all still there, but they can be disabled if using the
Agent_Terminal, a single "master tool" that cuts token usage by over 90%!Anthropic found 98.7% context savings using code execution with MCP, Cloudflare published similar findings. This is my open-source implementation of the same idea.
# The Problem
Traditional MCP exposes every tool definition directly to the model. With 12 tools, that's thousands of tokens consumed *before the conversation even starts*. Each tool call also passes intermediate results through the context window β a 10,000-row spreadsheet? That's all going into context just to sum a column.
# The Solution: One Tool to Rule Them All
Agent_Terminal wraps all 12 tools (Web_Search, Web_Fetch, File_System, Generate_Image, Generate_Speech, Generate_Video, Deep_Research, Memory_Manager, Obsidian_Vault, Shell_Command, Code_Interpreter) into a single Python code execution gateway.Instead of the model making individual tool calls, it writes Python code that orchestrates the tools directly:
# Search for Bitcoin price
result = Web_Search("current price of bitcoin", max_results=3)
print(result)Don't know what tools are available? The agent can discover them at runtime:
print(search_tools('image')) # Find tools by keyword
print(usage('Generate_Image')) # Get full docs for a specific toolThe individual direct tool calls are all still there, but they can be disabled if using the
Agent_Terminal. Try it now - https://www.nymbo.net/nymbot
- 1 reply
nroggendorffΒ
posted
an
update
about 1 month ago
Post
2843
I am now being charged for paused and unstarted spaces out of the blue.
I think this is it, folks. o7
The unstarted spaces I can get behind. I would've appreciated a warning email first, but whatever. However, every time I restart the active usage goes up, despite all of my spaces being moved to CPU (free), and being paused.
I think this is it, folks. o7
The unstarted spaces I can get behind. I would've appreciated a warning email first, but whatever. However, every time I restart the active usage goes up, despite all of my spaces being moved to CPU (free), and being paused.
-
- 13 replies
nroggendorffΒ
posted
an
update
about 1 month ago
Post
2361
Developing with ZeroGPU without a PRO account is painful. They give you so many requests at once, but then have like a 24 hour cooldown. I vote less requests in a batch, but then a shorter cooldown.
or just less of a cooldown, but i understand if that is not allowed
or just less of a cooldown, but i understand if that is not allowed
-
-
- 3 replies
Post
793
The #1 trending AI/ML dataset today π
Massive scale, diversity and end-to-end potential from nvidia !
nvidia/PhysicalAI-Autonomous-Vehicles
Massive scale, diversity and end-to-end potential from nvidia !
nvidia/PhysicalAI-Autonomous-Vehicles
Post
586
The new King πhas arrived!
Moonshot AI now the top model on Hugging Face π₯
moonshotai/Kimi-K2-Thinking
Moonshot AI now the top model on Hugging Face π₯
moonshotai/Kimi-K2-Thinking
Post
2731
πΈπ€You donβt need 100 GPUs to train something amazing!
Our Smol Training Playbook teaches you a better path to world-class LLMs, for free!
Check out the #1 trending space on π€ :
HuggingFaceTB/smol-training-playbook
Our Smol Training Playbook teaches you a better path to world-class LLMs, for free!
Check out the #1 trending space on π€ :
HuggingFaceTB/smol-training-playbook
DmitryRyuminΒ
posted
an
update
about 2 months ago
Post
1249
πποΈπ New Research Alert - ICCV 2025 (Poster)! πποΈπ
π Title: Is Less More? Exploring Token Condensation as Training-Free Test-Time Adaptation π
π Description: Token Condensation as Adaptation (TCA) improves the performance and efficiency of Vision Language Models in zero-shot inference by introducing domain anchor tokens.
π₯ Authors: Zixin Wang, Dong Gong, Sen Wang, Zi Huang, Yadan Luo
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: Is Less More? Exploring Token Condensation as Training-free Test-time Adaptation (2410.14729)
π Repository: https://github.com/Jo-wang/TCA
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the Session 1: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/session-1.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #TestTimeAdaptation #TokenCondensation #VisionLanguageModels #TrainingFreeAdaptation #ZeroShotLearning #EfficientAI #AI #ICCV2025 #ResearchHighlight
π Title: Is Less More? Exploring Token Condensation as Training-Free Test-Time Adaptation π
π Description: Token Condensation as Adaptation (TCA) improves the performance and efficiency of Vision Language Models in zero-shot inference by introducing domain anchor tokens.
π₯ Authors: Zixin Wang, Dong Gong, Sen Wang, Zi Huang, Yadan Luo
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: Is Less More? Exploring Token Condensation as Training-free Test-time Adaptation (2410.14729)
π Repository: https://github.com/Jo-wang/TCA
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the Session 1: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/session-1.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #TestTimeAdaptation #TokenCondensation #VisionLanguageModels #TrainingFreeAdaptation #ZeroShotLearning #EfficientAI #AI #ICCV2025 #ResearchHighlight
xianbaoΒ
authored
a
paper
about 2 months ago
DmitryRyuminΒ
posted
an
update
about 2 months ago
Post
2395
πποΈπ New Research Alert - ICCV 2025 (Oral)! πποΈπ
π Title: Diving into the Fusion of Monocular Priors for Generalized Stereo Matching π
π Description: The proposed method enhances stereo matching by efficiently combining unbiased monocular priors from vision foundation models. This method addresses misalignment and local optima issues using a binary local ordering map and pixel-wise linear regression.
π₯ Authors: Chengtang Yao, Lidong Yu, Zhidan Liu, Jiaxi Zeng, Yuwei Wu, and Yunde Jia
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: Diving into the Fusion of Monocular Priors for Generalized Stereo Matching (2505.14414)
π Repository: https://github.com/YaoChengTang/Diving-into-the-Fusion-of-Monocular-Priors-for-Generalized-Stereo-Matching
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the 3D Pose Understanding Section: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/3d-pose-understanding.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #StereoMatching #MonocularDepth #VisionFoundationModels #3DReconstruction #Generalization #AI #ICCV2025 #ResearchHighlight
π Title: Diving into the Fusion of Monocular Priors for Generalized Stereo Matching π
π Description: The proposed method enhances stereo matching by efficiently combining unbiased monocular priors from vision foundation models. This method addresses misalignment and local optima issues using a binary local ordering map and pixel-wise linear regression.
π₯ Authors: Chengtang Yao, Lidong Yu, Zhidan Liu, Jiaxi Zeng, Yuwei Wu, and Yunde Jia
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: Diving into the Fusion of Monocular Priors for Generalized Stereo Matching (2505.14414)
π Repository: https://github.com/YaoChengTang/Diving-into-the-Fusion-of-Monocular-Priors-for-Generalized-Stereo-Matching
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the 3D Pose Understanding Section: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/3d-pose-understanding.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #StereoMatching #MonocularDepth #VisionFoundationModels #3DReconstruction #Generalization #AI #ICCV2025 #ResearchHighlight
Post
1098
I've added an 11th tool to the
Nymbo/Tools MCP server, it's for your
# Obsidian_Vault Tool
It's basically a read-only version of the
Run the server locally and set the
# Web_Search Improvements
The
There's also a new date filter to limit results to those created in the past day, week, month, or year. Oh, and uhh, SafeSearch is now
# Web_Fetch Improvements
As context-efficient as the Markdown mode is for web browsing, sometimes it does lose important context in the conversion from HTML to Markdown. So I've added a new HTML mode to the
# A Note on Claude Skills
I've been having fun with the new
Happy building ~
Obsidian_Vault. I'd argue it's far more context-efficient than any other Obsidian MCP I've seen, and doesn't require any plugins. Also some big improvements to the Web_Search and Web_Fetch tools.# Obsidian_Vault Tool
It's basically a read-only version of the
File_System tool, but it works so well for navigating Obsidian without unnecessary context. It supports recursive (full-text) search across the entire vault, and supports offset so the agent can "scroll" through a document without re-consuming tokens.Run the server locally and set the
OBSIDIAN_VAULT_ROOT environment variable to your vault's root path. If you don't use Obsidian, this is perfectly usable as simply a read-only filesystem.# Web_Search Improvements
The
Web_Search tool previously just used DuckDuckGo as a backend search engine, but now it also supports Bing, Brave, Yahoo, and Wikipedia. Default engine is auto which provides results from all backends in recommended order. Still doesn't require any kind of API or auth for Web_Search.There's also a new date filter to limit results to those created in the past day, week, month, or year. Oh, and uhh, SafeSearch is now
off by default :)# Web_Fetch Improvements
As context-efficient as the Markdown mode is for web browsing, sometimes it does lose important context in the conversion from HTML to Markdown. So I've added a new HTML mode to the
Web_Fetch tool that basically executes a cURL request on the URL, returning the full HTML page if necessary.# A Note on Claude Skills
I've been having fun with the new
File_System and Shell_Command tools. Using Claude Skills doesn't currently work in the public HF space because of environment restrictions, but using Skills works perfectly well running locally.Happy building ~
Build Failure (apt-get permission denied) in Dev Mode on Gradio + ZeroGpu Space
4
#26 opened about 2 months ago
by
Archime
DmitryRyuminΒ
posted
an
update
about 2 months ago
Post
2832
πππ New Research Alert - ICCV 2025 (Oral)! ππ€π
π Title: Understanding Co-speech Gestures in-the-wild π
π Description: JEGAL is a tri-modal model that learns from gestures, speech and text simultaneously, enabling devices to interpret co-speech gestures in the wild.
π₯ Authors: @sindhuhegde , K R Prajwal, Taein Kwon, and Andrew Zisserman
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: Understanding Co-speech Gestures in-the-wild (2503.22668)
π Web Page: https://www.robots.ox.ac.uk/~vgg/research/jegal
π Repository: https://github.com/Sindhu-Hegde/jegal
πΊ Video: https://www.youtube.com/watch?v=TYFOLKfM-rM
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the Human Modeling Section: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/human-modeling.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #CoSpeechGestures #GestureUnderstanding #TriModalRepresentation #MultimodalLearning #AI #ICCV2025 #ResearchHighlight
π Title: Understanding Co-speech Gestures in-the-wild π
π Description: JEGAL is a tri-modal model that learns from gestures, speech and text simultaneously, enabling devices to interpret co-speech gestures in the wild.
π₯ Authors: @sindhuhegde , K R Prajwal, Taein Kwon, and Andrew Zisserman
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: Understanding Co-speech Gestures in-the-wild (2503.22668)
π Web Page: https://www.robots.ox.ac.uk/~vgg/research/jegal
π Repository: https://github.com/Sindhu-Hegde/jegal
πΊ Video: https://www.youtube.com/watch?v=TYFOLKfM-rM
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the Human Modeling Section: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/human-modeling.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #CoSpeechGestures #GestureUnderstanding #TriModalRepresentation #MultimodalLearning #AI #ICCV2025 #ResearchHighlight
DmitryRyuminΒ
posted
an
update
about 2 months ago
Post
3958
ππ‘π New Research Alert - ICCV 2025 (Oral)! ππͺπ
π Title: LoftUp: Learning a Coordinate-based Feature Upsampler for Vision Foundation Models π
π Description: LoftUp is a coordinate-based transformer that upscales the low-resolution features of VFMs (e.g. DINOv2 and CLIP) using cross-attention and self-distilled pseudo-ground truth (pseudo-GT) from SAM.
π₯ Authors: Haiwen Huang, Anpei Chen, Volodymyr Havrylov, Andreas Geiger, and Dan Zhang
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models (2504.14032)
π Github Page: https://andrehuang.github.io/loftup-site
π Repository: https://github.com/andrehuang/loftup
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the Foundation Models and Representation Learning Section: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/foundation-models-and-representation-learning.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #LoftUp #VisionFoundationModels #FeatureUpsampling #Cross-AttentionTransformer #CoordinateBasedLearning #SelfDistillation #PseudoGroundTruth #RepresentationLearning #AI #ICCV2025 #ResearchHighlight
π Title: LoftUp: Learning a Coordinate-based Feature Upsampler for Vision Foundation Models π
π Description: LoftUp is a coordinate-based transformer that upscales the low-resolution features of VFMs (e.g. DINOv2 and CLIP) using cross-attention and self-distilled pseudo-ground truth (pseudo-GT) from SAM.
π₯ Authors: Haiwen Huang, Anpei Chen, Volodymyr Havrylov, Andreas Geiger, and Dan Zhang
π Conference: ICCV, 19 β 23 Oct, 2025 | Honolulu, Hawai'i, USA πΊπΈ
π Paper: LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models (2504.14032)
π Github Page: https://andrehuang.github.io/loftup-site
π Repository: https://github.com/andrehuang/loftup
π ICCV-2023-25-Papers: https://github.com/DmitryRyumin/ICCV-2023-25-Papers
π Added to the Foundation Models and Representation Learning Section: https://github.com/DmitryRyumin/ICCV-2023-25-Papers/blob/main/sections/2025/main/foundation-models-and-representation-learning.md
π More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
π Keywords: #LoftUp #VisionFoundationModels #FeatureUpsampling #Cross-AttentionTransformer #CoordinateBasedLearning #SelfDistillation #PseudoGroundTruth #RepresentationLearning #AI #ICCV2025 #ResearchHighlight