datalyes/title2full
Viewer
ā¢
Updated
ā¢
18.7k
ā¢
1
iliass ayaou
datalyes
AI & ML interests
information retrieval, patent retrieval, knowledge management, data engineering and architecture, NLP
Recent Activity
posted
an
update
1 day ago
I am happy to share that three models from PATENTEB are now publicly available: patembed-large, patembed-base, and patembed-base_long_4096.
The models can be found here:
https://huggingface.co/collections/datalyes/patembed-models-collection
Feedback, issues, and use cases are very welcome.
new activity
about 1 month ago
datalyes/DAPFAM_patent:Add task category: text-retrieval
reacted
to
nouamanetazi's
post
with š
3 months ago
After training šš¦šØš„ššš on ššš ššššš¬ for nearly a month, I've come to realize something most people overlook: š¢š§šš«šš¬šš«š®ššš®š«š š¢š¬ šš”š š¦šš¤š-šØš«-šš«ššš¤ šššššØš« š¢š§ ššš šš«šš¢š§š¢š§š . š„
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious šššš šš«š«šØš«š¬, or when your expensive GPU cluster is running at šš% šššš¢šš¢šš§šš², the problem isn't your model. It's most probably a š¦š¢š¬š®š¬š šØš šš”š š”šš«šš°šš«š. š ļø
Questions that seemed simple but had no clear answers: Why is ššØš šš«šš¢š§š¢š§š š¬š„šØš°šš« šš”šš§ ššš§š¬š š¦šØššš„š¬? Which šššš šš„šš š¬ should we actually set? How often should we checkpoint without killing throughput?
That's why we built šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤ š: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the š¢š§šš«šš¬šš«š®ššš®š«š š„šš²šš« that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: šššš š”š¢ššš¢š§š š šš/š¬, šššš¢š§š¤ š.š š«šššš”š¢š§š ššš šš/š¬, šššš ššš§š šš šš.š šš/š¬. Then we ran collective operations across ššš šššš¬ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ššš šš/š¬ on a single node to ššš-ššš šš/š¬ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤: https://lnkd.in/e5MKXUHS
Shared with ā¤ļø by the HuggingFace team
Organizations
