推文
760 tweets · 188 sources账号:全部mattpocockukdair_aisimonwbchernyadocompletetrq212dexhorthyyetonefelixrieseberg0xblacklightdani_avila7zarazhangruiAlchainHustbadlogicgamesdoteypetergyangvikingmuteryancarsonClaudeDevskunchenguidleon7haoctatedeviamzhihuikieranklaassenmckaywrigleylennysanthdxrRhysSullivangarrytanjakevin7johnlindquistkarpathyservasyy_aitheo0xMovez9hillsRLanceMartinaidenybaidingyihylarucodermitchellhmitsuhikomvanhornomarsar0steipetezeegGeminiAppGergelyOroszZaynHaoaddyosmaniantirezdanshipperdillon_mulroyelonmuskewind_devidoubiccnummanalirealWeZZardswyxthsottiauxyiliush0xPaulius0x_rodyAYi_AInotesAnthropicAIBarret_ChinaBenJames_____CMGS1988ChadMoranDanielMiesslerDaveJDavidKPianoDimillianFactoryAIFardeemMFarzaTVGoogleDeepMindHamelHusainHiTw93HilaShmuelIfanJewIndieDevHaileyJack_W_LindseyJackywineJasonZXJiaxi_CuiJinjingLiangKhalidWarsaLinearUncleLinghuaJMatthewBermanMaxForAIMeari_V2_0_GMnilaxQingQ77RaillyHugoSaccc_cTaniyatweets_VincentLogicai_explorer25aibuilderclub_alexalbert__alliekmillerandrewfarahantoinecojpanxue201arkuy99artmanasynkimobbssppllvvbearliubentossellbridgemindaibuaaxhmcailynyongyongcgtwtscharmaine_kleechrisbarberchrisparkXclairevodelba_oliveirademishassabisdoodlesteindriaforallelithrarelvissunericzakariassonfrancoisfleuretgabriell_labiamsahaj_xyzimdigitalashishimvihvjarredsumnerjerryjliu0jianshuojiayuan_jyjlongsterjonas_nellejoshalbrechtjpschroederkaiofreitaskepanolexruslogancyangmattypnabeelnifinetninthbit_aioran_gepatrickcpejmanjohnpetradonkaprathamgrvprathyvshquanruzhuoxiuquant_sheepraroquerealCaigurepsiacerohanpaul_aisamasamuelstroscheisawyerhoodshadcnshao__mengshaogefenhaoshreyansjtechgirl1908techwith_ramthorstenballtianyitobiaswuptoddsaunderstricaltturingoututuretomuniswap12untraceable_thevanillaCitronvelvet_sharkwengtianxinwquguruxiaogaifunxicilionycombinatorzachlloydtweetsziwenxu_zodchiii
@zeeg·6月18日For You
I'd love to see examples of a great eval setup that runs against a complex agent.
We all know that asserting input prompt to output prompt isnt testing anything, but I'm curious how everyone actually checks behavior.
Things like: did the agent make this tool call? did it hit this network endpoint? did it write to this data store?
Obviously some of those are determnistic, but some come from actual behavior within the loop.
我很想看到针对复杂 agent 的优秀 eval 设置案例。 大家都知道，断言「输入 prompt → 输出 prompt」根本没在测任何东西，但我很好奇大家实际上是怎么检验行为的。 比如：agent 有没有发出这个 tool call？有没有命中这个网络端点？有没有写入这个数据存储？ 其中有些是确定性的，但有些源自 loop 内部的真实行为。
@zeeg·6月12日For You
codex writes the most digusting code
idk who's responsible for pre-training over there but you gotta flip the script https://t.co/38I6RWIStu
Codex 写出了最让人难受的代码 不知道他们预训练那边谁负责，但真的得改改思路了
@zeeg·6月8日For You
"you should be running rabbitmq and piping tasks into it and so your agent can generate max slop while you pretend your output is valuable"
im so tired of hearing about "groundbreak techniques" to produce more unmaintainable software
「你应该跑 RabbitMQ，把任务往里面管，这样你的 agent 就能产出最大量的 slop，而你假装自己的输出很有价值」 我实在听腻了那些「突破性技术」，结果不过是产出更多难以维护的软件。
760 tweets · 188 sources