琥珀青葉@KohakuLab @KBlueleaf

For those who want to argue compute itself cost more energy: the memory system in RTX5090 can occupy 300 watt when the whole card consume 575watt from power supply. I have measured this. If you make tensor core FULLY LOADED it is 320watt, and in training the tensor and cuda core never fully loaded. but whole card still go to 575 watt. in gpu-z you may get a more accurate number and iirc it is around 320~360watt.

Yann LeCun @ylecun

2 days ago

@francoisfleuret Moving bits to and from memory. Because of parasitic capacitance+resistance of the wires. The bigger the memory, the longer the wires. The main trick is to organize the memory hierarchically: registers, small on-chip SRAM, caches of various types, and external RAM. It's all

24 34 869 76K 268

1 0 13 2K 6

View Details

琥珀青葉@KohakuLab @KBlueleaf

5 days ago

不是，沒有公司同意就用公司的商標只是被開除已經是最輕的下場了吧? Google是不是量產這種搞不清情況然後被開除就是"公司害怕被顛覆"的傻逼啊。之前有個搞所謂AI安全還AI倫理的也是，發論文抨擊什麼AI倫理什麼AI bias的東西，沒有經過公司同意就發論文(note根據他本人自己給的資訊，公司壓根就不知道她想發這個論文，所以沒有什麼「公司禁止她發所以她才自行發布」，而且她發的時候affilation還標了公司)，然後被約談開除，然後就大肆渲染是Google不想搞AI ethic/DEI blablabla才開除他的。但根據她自己說的前因後果，她想"呼籲"有關AI ethic跟AI bias相關的東西，嗯她自己根本就沒找公司裡其他人談過，呼籲了個der。像這個也是，google員工搞google自家平台的開源工具，用google的商標、logo。結果呢? google在這東西走紅之前壓根就不知道這東西的存在? bro，我重申一遍，這只是被開除已經是最最最輕的結果了，不用上法庭打必輸的官司已經是google給的最後體面了。我以為是個人都知道公司員工想要做任何與公司相關的開源項目以及使用公司的任何商標或名稱都必須經過公司法務的確認與團隊上司的同意呢==，怎麼，我在跟蘋果還有NV合作的時候跑的那幾個月流程都是我白跑了嗎? 利申: 本人GitHub 7k star、HF 3M download、Pypi 6M download開源作者，同時是100k star repo的core dev之一，應該算夠支持開源了吧。

0 0 32 1K 3

View Details

琥珀青葉@KohakuLab @KBlueleaf

5 days ago

@yeh60123 平均名次臺1 清2.5 交3 成3.5 (?)

1 0 1 109 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

6 days ago

the adaLN is powerful is bcuz it provide different basis in gate/scale/shift if you even share the basis then it is pretty meaningless to do it (note, when all of them share same basis it is linear on that, but shared adaLN is not linear on any basis bcuz the gate/shift/scale have different basis comes from the initial projector)

0 0 1 19 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

6 days ago

bcuz just shared adaLN have shrinked the param a lot and basically no degradataion. BTW in shared adaLN you NEVER have (dim, 6*dim) but (6*dim) it is large bcuz at before every ppl put standalone linear (dim, 6*dim appear here) in every single block but in shared adaLN you only have single (dim, 6*dim) which is ignorable and wan-series models is more aggressive than this (dim, dim) at first to obtain the single token then (dim, 6) per block to do scaling BUT^ this is actually very very bad bcuz it even share the same basis for different block's gate/shift/scale I will say if you WANT adaln, then just shared 6*dim single projector If you still feel this is too large, just DON"T use adaln at all, the wan's design never makes sense and that's why it is not popular

3 0 2 43 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

6 days ago

@Kenntnis22 本(敝) <> 貴、職、准駁語、請示語、期望或目的語這些在稍微正式一點的文書、書信往來裡面非常非常常用。但確實不是整張表都很常用。

0 0 9 1K 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

6 days ago

@YouJiacheng Also shared adaLN indeed basically affect nothing in compute (bcuz the removed linear only works on single token) so yes it is good iso-compute baseline.

1 0 1 41 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

6 days ago

@YouJiacheng Yeah shared adaLN works very well no matter in research or in real world model (like Chroma is the shared adaLN mod ver of Flux) For time as token iirc it works well but HOW to make that token will affect a lot (you want it to have similar statistics property with xt)

1 0 3 38 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

7 days ago

@dearemon btw這題目有限制只能一次跑5匹馬不然25匹馬跑一場就搞定了

1 0 0 354 0

View Details

琥珀青葉@KohakuLab @KBlueleaf

7 days ago

@dearemon 5場找出每5匹馬的最快。再一場找出這5個各組最快馬的順序: 1. -> 全場最快第一名同組的23 + 第二名與同組的2 + 第3名再跑一場前2就是全場第二第三總共七場解決 (各組最快裡面的第一名同組的2/3可能全場2/3 第二名同組的2有可能全場3 第3名有可能全場3，所以1-2 1-3 2-1 2-2 3-1還要再跑一場)