Daniel Lemire's blog
Programmer time and the pitfalls of wasteful work
Programmer time is precious. This realization should shape our approach to software development, focusing our efforts on tasks that genuinely contribute to the improvement of our code and the software ecosystem.
What does matter?
1. 1. Hunting for bugs. I like to add tests, and then even more tests. The time spent building tests should proportionate to the time spent building the software. Fuzzing is also fantastically useful. I love using sanitizers.
2. Fixing bugs. Bugs disrupt user experience, compromise functionality, and can even introduce security vulnerabilities. Addressing bugs is critical to build trust in the software.
3. Documentation matters. Underdocumented code is mysterious and may trigger unnecessary surprises. Lack of documentation may also harm relationships with users.
4. Adding new features. Innovation and growth in software come from introducing new features. Features should be user visible: ‘internal’ features are often wasteful.
5. Improving Performance. Performance enhancement is all about making the software run faster, use fewer resources, or handle larger workloads more efficiently. This can significantly impact user satisfaction, particularly in applications where speed is paramount. Improving performance is not about identify bottlenecks… it is an ongoing journey. You need a good design and multiple rounds of optimizations. You can often continue to improve the performance for years and years.
However, there are areas where I believe our time is not well spent:
● Patching code to silence false positives from disabled-by-default static analyzers. The level 4 warnings under Visual Studio when compiling C++ code is a good example, but so are the obscure GCC and clang warnings. Static analyzers are tools that can scan code for potential issues without executing the program. However, when these tools are overly strict or misconfigured, they might report numerous false positives; issues that aren’t actually problems. Spending time patching code merely to quiet these false alarms is, in my view, wasteful. It diverts attention from more impactful work. This is not to say that static analysis is not beneficial; when used correctly, it can save considerable time and resources. But the effort required to address non-issues can quickly become counterproductive.
● Aimless refactoring is also often wasteful. Renaming classes, moving code around just so that it looks ‘nice’. I am not against the occasional cleaning round… but it is should not be time consuming. Refactoring for its own sake may become an excuse for not fixing bugs or for not improving the performance. It is easy work, but often not impactful.
While we strive for perfection in our code, we must also be strategic about where we invest our most precious resource: programmer time. Let us prioritize what truly matters in the grand scheme of software development.
source
Programmer time and the pitfalls of wasteful work
Programmer time is precious. This realization should shape our approach to software development, focusing our efforts on tasks that genuinely contribute to the improvement of our code and the software ecosystem.
What does matter?
1. 1. Hunting for bugs. I like to add tests, and then even more tests. The time spent building tests should proportionate to the time spent building the software. Fuzzing is also fantastically useful. I love using sanitizers.
2. Fixing bugs. Bugs disrupt user experience, compromise functionality, and can even introduce security vulnerabilities. Addressing bugs is critical to build trust in the software.
3. Documentation matters. Underdocumented code is mysterious and may trigger unnecessary surprises. Lack of documentation may also harm relationships with users.
4. Adding new features. Innovation and growth in software come from introducing new features. Features should be user visible: ‘internal’ features are often wasteful.
5. Improving Performance. Performance enhancement is all about making the software run faster, use fewer resources, or handle larger workloads more efficiently. This can significantly impact user satisfaction, particularly in applications where speed is paramount. Improving performance is not about identify bottlenecks… it is an ongoing journey. You need a good design and multiple rounds of optimizations. You can often continue to improve the performance for years and years.
However, there are areas where I believe our time is not well spent:
● Patching code to silence false positives from disabled-by-default static analyzers. The level 4 warnings under Visual Studio when compiling C++ code is a good example, but so are the obscure GCC and clang warnings. Static analyzers are tools that can scan code for potential issues without executing the program. However, when these tools are overly strict or misconfigured, they might report numerous false positives; issues that aren’t actually problems. Spending time patching code merely to quiet these false alarms is, in my view, wasteful. It diverts attention from more impactful work. This is not to say that static analysis is not beneficial; when used correctly, it can save considerable time and resources. But the effort required to address non-issues can quickly become counterproductive.
● Aimless refactoring is also often wasteful. Renaming classes, moving code around just so that it looks ‘nice’. I am not against the occasional cleaning round… but it is should not be time consuming. Refactoring for its own sake may become an excuse for not fixing bugs or for not improving the performance. It is easy work, but often not impactful.
While we strive for perfection in our code, we must also be strategic about where we invest our most precious resource: programmer time. Let us prioritize what truly matters in the grand scheme of software development.
source
频道主刚在玩这个，并获得第一题的红包，感觉好玩，哈哈
2025 乙巳蛇年的新年红包
https://hb.lohu.info
1. 这是每年春节 Soha 的传统节目，是一个解密寻宝游戏（a.k.a. CTF），利用你的知识（现学大概也是足够的）解决所有题目，获得红包口令，口令可进入支付宝领取红包。
2. 你应该需要电脑才能愉快玩耍，但手机可能也能解一部分。不涉及任何暴力解法（爆破等）。
3. 本次活动时间从北京时间 2025 年 1 月 28 日 20 时开始，持续 24 个小时。如果红包被提前领完不会补发。
4. 这个游戏由 Soha 制作，在游戏结束后将在我的博客放出题解，往年的内容也可以在博客上找到。更多提示请在活动页面查看。
5. 最后祝大家，新年快乐！
如有疑问可以私聊 @sohajin 提出。
https://hb.lohu.info
1. 这是每年春节 Soha 的传统节目，是一个解密寻宝游戏（a.k.a. CTF），利用你的知识（现学大概也是足够的）解决所有题目，获得红包口令，口令可进入支付宝领取红包。
2. 你应该需要电脑才能愉快玩耍，但手机可能也能解一部分。不涉及任何暴力解法（爆破等）。
3. 本次活动时间从北京时间 2025 年 1 月 28 日 20 时开始，持续 24 个小时。如果红包被提前领完不会补发。
4. 这个游戏由 Soha 制作，在游戏结束后将在我的博客放出题解，往年的内容也可以在博客上找到。更多提示请在活动页面查看。
5. 最后祝大家，新年快乐！
如有疑问可以私聊 @sohajin 提出。
Chips and Cheese
Inside SiFive’s P550 Microarchitecture
#ChipAndCheese
Telegraph | source
(author: George Cozma)
Inside SiFive’s P550 Microarchitecture
#ChipAndCheese
Telegraph | source
(author: George Cozma)
Daniel Lemire's blog
Regular expressions can blow up!
Regular expressions, often abbreviated as regex, are a powerful tool for pattern matching within text. For example, the expression
would match a positive number such as 1.1 or 12. If designed and tested with care, regular expressions may be used in mission-critical software. However, their power comes with a risk: it is possible to design small regular expressions that are very expensive to run on even small strings.
To make matters more complicated, there are several regular-expression engines, and they differ in their syntax and implementation. Let me consider the regular-expression engine used by the C++ language under Linux (libgc++).
Consider the following program. It uses the string “Everyone loves Lucy.” and d a regex pattern (.*+s}}@w. I am not exactly sure what this pattern is supposed to do, but it is accepted by the engine. The program then uses std::regex_search to look for matches of this pattern within the string, storing potential matches in a std::smatch object, and outputs whether a match was found or not.
Using GCC 12 and a recent Linux server, this program takes about 7 minutes to run.
In other words, a bad regular expression can crash your systems. It is not just theoretical, the Cloudflare corporation suffered a major outage in 2019 due to a bad regular expression.
Use regular expressions with care.
source
Regular expressions can blow up!
Regular expressions, often abbreviated as regex, are a powerful tool for pattern matching within text. For example, the expression
\d*\.?\d+would match a positive number such as 1.1 or 12. If designed and tested with care, regular expressions may be used in mission-critical software. However, their power comes with a risk: it is possible to design small regular expressions that are very expensive to run on even small strings.
To make matters more complicated, there are several regular-expression engines, and they differ in their syntax and implementation. Let me consider the regular-expression engine used by the C++ language under Linux (libgc++).
Consider the following program. It uses the string “Everyone loves Lucy.” and d a regex pattern (.*+s}}@w. I am not exactly sure what this pattern is supposed to do, but it is accepted by the engine. The program then uses std::regex_search to look for matches of this pattern within the string, storing potential matches in a std::smatch object, and outputs whether a match was found or not.
#include <iostream>
#include <regex>
int main() {
std::string text = "Everyone loves Lucy.";
std::regex pattern(R"(.*+s}}@w)");
// Perform regex search
std::smatch match;
bool found = std::regex_search(text, match, pattern);
std::cout << "Regex search result: "
<< (found ? "Match found" : "No match") << std::endl;
return 0;
}Using GCC 12 and a recent Linux server, this program takes about 7 minutes to run.
In other words, a bad regular expression can crash your systems. It is not just theoretical, the Cloudflare corporation suffered a major outage in 2019 due to a bad regular expression.
Use regular expressions with care.
source
Chips and Cheese
Disabling Zen 5’s Op Cache and Exploring its Clustered Decoder
#ChipAndCheese
Telegraph | source
(author: Chester Lam)
Disabling Zen 5’s Op Cache and Exploring its Clustered Decoder
#ChipAndCheese
Telegraph | source
(author: Chester Lam)
Matt Keeter
Guided by the beauty of our test suite
source
(author: Matt Keeter (matt.j.keeter@gmail.com))
Guided by the beauty of our test suite
source
(author: Matt Keeter (matt.j.keeter@gmail.com))
Beating cuBLAS in Single-Precision General Matrix Multiplication https://salykova.github.io/sgemm-gpu
Chips and Cheese
Inside the AMD Radeon Instinct MI300A's Giant Memory Subsystem
#ChipAndCheese
Telegraph | source
(author: Chester Lam)
Inside the AMD Radeon Instinct MI300A's Giant Memory Subsystem
#ChipAndCheese
Telegraph | source
(author: Chester Lam)
Arch Linux: Recent news updates
Critical rsync security release 3.4.0
We'd like to raise awareness about the rsync security release version
An attacker only requires anonymous read access to a vulnerable rsync server, such as a public mirror, to execute arbitrary code on the machine the server is running on. Additionally, attackers can take control of an affected server and read/write arbitrary files of any connected client. Sensitive data can be extracted, such as OpenPGP and SSH keys, and malicious code can be executed by overwriting files such as
We highly advise anyone who runs an rsync daemon or client prior to version
All infrastructure servers and mirrors maintained by Arch Linux have already been updated.
source
(author: Robin Candau)
Critical rsync security release 3.4.0
We'd like to raise awareness about the rsync security release version
3.4.0-1 as described in our advisory ASA-202501-1.An attacker only requires anonymous read access to a vulnerable rsync server, such as a public mirror, to execute arbitrary code on the machine the server is running on. Additionally, attackers can take control of an affected server and read/write arbitrary files of any connected client. Sensitive data can be extracted, such as OpenPGP and SSH keys, and malicious code can be executed by overwriting files such as
~/.bashrc or ~/.popt.We highly advise anyone who runs an rsync daemon or client prior to version
3.4.0-1 to upgrade and reboot their systems immediately. As Arch Linux mirrors are mostly synchronized using rsync, we highly advise any mirror administrator to act immediately, even though the hosted package files themselves are cryptographically signed.All infrastructure servers and mirrors maintained by Arch Linux have already been updated.
source
(author: Robin Candau)
Chips and Cheese
Nvidia's Announcements at CES 2025
#ChipAndCheese
Telegraph | source
(author: George Cozma)
Nvidia's Announcements at CES 2025
#ChipAndCheese
Telegraph | source
(author: George Cozma)
Daniel Lemire's blog
The ivory tower’s drift: how academia’s preference for theory over empiricism fuels scientific stagnation
Almost all of academic science has moved away from actual (empirical) science. It is higher status to work on theories and models. I believe that it is closely related to well documented scientific stagnation as theory is often ultimately sterile.
This tendency is quite natural in academia if there is no outside pressure… And is the main reason why academia should be ruthlessly judged by practitioners and users. As soon as academia can isolate itself in a bubble, it is bound to degrade.
It is worth trying to understand some of the factors driving this degradation… Theoretical work can sometimes be seen as more complex. This complexity can be mistakenly equated with higher intelligence or prestige. Empirical work, while also complex, often deals with tangible, observable data, which might seem more straightforward to the uninitiated.
Empirical work is more likely to lead to nuanced or inconclusive results while theory is often seemingly more direct and definitive. Theoretical research often requires fewer resources than large-scale empirical studies which might need extensive funding for equipment, data collection, and personnel. Thus you get to do more research with less using models and theory.
Theoretical work is often seen as requiring a high level of creativity to devise new frameworks or models. While empirical work also requires creativity in design, execution, and interpretation, the creativity in data collection or experimental design might be less recognized or appreciated.
The educational system often glorifies theoretical knowledge over practical skills until one reaches higher education or specialized training. E.g., we eagerly make calculus compulsory even if it has modest relevance in most practical fields. This educational bias can carry over into professional work.
Society must demand actual results. We must reject work that is said ‘to improve our understanding’ or ‘to lay a foundation for further work’. We must demand cheaper rockets, cures for cancer, software that is efficient. As long as academic researchers are left to their own devices, they will continue to fill the minds of the young with unnecessary models. They must be held accountable.
source
The ivory tower’s drift: how academia’s preference for theory over empiricism fuels scientific stagnation
Almost all of academic science has moved away from actual (empirical) science. It is higher status to work on theories and models. I believe that it is closely related to well documented scientific stagnation as theory is often ultimately sterile.
This tendency is quite natural in academia if there is no outside pressure… And is the main reason why academia should be ruthlessly judged by practitioners and users. As soon as academia can isolate itself in a bubble, it is bound to degrade.
It is worth trying to understand some of the factors driving this degradation… Theoretical work can sometimes be seen as more complex. This complexity can be mistakenly equated with higher intelligence or prestige. Empirical work, while also complex, often deals with tangible, observable data, which might seem more straightforward to the uninitiated.
Empirical work is more likely to lead to nuanced or inconclusive results while theory is often seemingly more direct and definitive. Theoretical research often requires fewer resources than large-scale empirical studies which might need extensive funding for equipment, data collection, and personnel. Thus you get to do more research with less using models and theory.
Theoretical work is often seen as requiring a high level of creativity to devise new frameworks or models. While empirical work also requires creativity in design, execution, and interpretation, the creativity in data collection or experimental design might be less recognized or appreciated.
The educational system often glorifies theoretical knowledge over practical skills until one reaches higher education or specialized training. E.g., we eagerly make calculus compulsory even if it has modest relevance in most practical fields. This educational bias can carry over into professional work.
Society must demand actual results. We must reject work that is said ‘to improve our understanding’ or ‘to lay a foundation for further work’. We must demand cheaper rockets, cures for cancer, software that is efficient. As long as academic researchers are left to their own devices, they will continue to fill the minds of the young with unnecessary models. They must be held accountable.
source
