The analysis and synthesis approaches to understanding systems have respectively been the driving forces for two major breakthroughs in 20th century physics: reductionist and emergent phenomena. Reductionism aims to understand a system by reducing it to component parts and assuming that the composition is simple. This attitude drives particle physics. On the other hand, one of the most important themes from condensed matter physics has been how "more is different" and collective phenomena can induce emergent behaviour which is very different from the behaviour of the constituent parts. In this perspective, the precise constituent parts don't really matter too much -- many substrates which like completely different to analysis can end up looking very similar in synthesis. This is the principle behind universality classes in critical phenomena. This patter of thought should also be familiar to folks who advocate for a "systems perspective".
In the language of differential vs integral calculus, you can have perfectly well behaved and physical functions whose derivatives can completely miss the global behaviour of the function i.e. smooth but not analytic eg exp(-1/x) at x=0. Funnily enough, this is exactly the form of the action taken by quantum mechanics and an argument for how the classical limit irrecoverably ignores the physics of quantum systems.
I loved reading this article. It was reasonably short to not make the reader lose interest. It jumped around in different domains to make a core point at the end about SREs managing complex systems. This gets even more difficult with the rate of change each of those systems has with coding agents. I don't know if I agree with the terms "synthesis" and "analysis" as the equivalent of global and local respectively, but it was a great read.
This post brought to mind a post by Bret Victor titled Up and Down the Ladder of Abstraction [1] where he frames working on and understanding problems at different layers of focus as providing insight that isn’t otherwise so apparent. It’s one of those posts that immediately resonated and that I’ve tried to internalize and adopt when approaching problems, especially those in new domains.
If my understanding of Kant is correct, then all analysis (i.e. taking things apart) is synthesis (i.e. putting things together) in reverse, and therefore the latter is a necessary prerequisite of the former. According to him, our minds very first brush with the real world (whatever that might be) is in the form of the "manifold" of raw sensory signals sans any form whatsoever (not even spatial or temporal arrangement), and the only way we can make sense of that manifold of signals or data is to reduce them through a process of combination by commonalities and separation by lack thereof. All objects, categories and concepts ultimately spring from this process of synthesis, some of it involuntary (e.g. the arrangement of certain inputs into three dimensional space) and some of it volitional (e.g. the positing of concepts and categories). Analysis is taking these constructs apart, much like one takes apart a mechanical watch. Except that the exercise tells us more about how our brain puts things together than what the thing actually is.
Taking this statement at face value, it means something in computer science: computing an answer (synthesis) is currently believed to be harder than checking an answer (analysis).
The simplest example to illustrate the claim is that factoring a number is harder than multiplying the two factors to check that it equals the original number, or even to decide whether the original is prime or composite (but without yielding the factors).
This also cuts to the heart of what NP means - it means that the answer to a yes/no problem about binary strings can be checked in polynomial time. It doesn't give a recipe for how to generate the answer, but it is implied that finding an answer can take up to exponential time and no more.
>incident response is one area where we are frequently confronted with synthesis problems: we have to understand how the pieces normally fit together in order to make sense of what is currently going wrong.
I'm not sure I can accept without further clarification equating that with synthesis. Normally in incident response, you start with a full system where in principle any root cause is possible and gradually reduce scope until finding the solution.
For me integration could be used for say taking three adhoc solutions and designing a general one that fits all, or merging two different services after a company merge. But anything debugging is very clearly analytic.
I get the point of the author about differentiation and integration. But, I did not follow completely how he connected it to analysis and synthesis. I mean, there isn't necessarily a one-to-one mapping between analysis and differentiation and synthesis and integration, right?
Maybe something like: When you do derivation you analyze how the function behaves at a given point. Whenever you analyze something you analyze a given thing, not all things. When you do integration you look at a function and then produce something that describes the function on every point. You come up with a closed integral-formula for the shole function. You need to synthesize something that describes the function everywhere. But when you derivate the dericate, you also produce a closed function that describes the function everywhere. Oops, I think I got lost here too.
Understanding existing code or generating possible solutions is getting easier. The hard part is still deciding what to build, what not to build, and how independent ideas fit together into something coherent.
In my experience, AI reduced the cost of implementation far more than it reduced the cost of synthesis.
As someone who's always loved synthesizing ideas and having lightbulb moments, I find the headline flattering. I wonder, though, if there's any more rigorous and general analysis (hah!) of the complexity of these two modes of thought.
EDIT: Fable 5 turned up some relevant references:
[1] Richardson, D. (1968). "Some Undecidable Problems Involving Elementary Functions of a Real Variable." Journal of Symbolic Logic, 33(4). — Differentiation is a simple recursive algorithm; deciding integrability in elementary terms is undecidable in general.
[2] Risch, R. H. (1969). "The Problem of Integration in Finite Terms." Transactions of the American Mathematical Society, 139. — The partial recovery: a (semi-)decision procedure for a restricted class.
[3] Guilford, J. P. (1967). The Nature of Human Intelligence. McGraw-Hill. — The divergent vs. convergent thinking distinction, the classic psychometric cousin of synthesis vs. analysis.
[4] Anderson, L. W., & Krathwohl, D. R. (Eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing (revision of Bloom's taxonomy). Longman. — Moved "Create" (synthesis) to the top of the cognitive hierarchy, above "Analyze."
[5] Aaronson, S. (2011). "Why Philosophers Should Care About Computational Complexity." arXiv:1108.1791. — Argues complexity asymmetries (verification vs. generation, P vs. NP) bear directly on questions about cognition.
The point I was waiting for but never came, so I'll make it here, is: Will LLMs be able to synthesize? Sure they can learn calculi. Lambda, differential, whatnot. But will Claude eventually be able to come up with a genuinely new formalism? A new calculus? Sth that allows us to do things we couldn't do before?
I don’t think the word applies to a blog post which its self-described as “ramblings”. The formula’s there to look scary and illustrate a point, nobody’s using it to integrate a Gaussian.
> And it turns out that it’s quite straightforward to calculate a derivative, no matter what type of function it is.
I get the author's point but this is not completely true; there exist functions that are not differentiable at certain places (e.g. ideal square waves) and others that are not differentiable anywhere (e.g. Weierstrass functions).
It's important to know that (in the usual setting of analysis) not every function is everywhere (or even anywhere) differentiable, but this is more orthogonal to the author's point than opposed to it. A square wave is piecewise differentiable and you can compute a piecewise derivative. The Weierstrass function is defined by an infinite series, and you can compute its derivative term-by-term by the usual rules and check that the result does not converge; it is indeed straightforward to calculate its nonexistent derivative, and this is what Weierstrass did!
In general, to even ask what it means to compute a derivative we need to specify some input language which describes functions in finite terms; we are necessarily in the world of constructions rather than (say) arbitrary set-theoretical maps between infinite sets. With this in mind, the claim that differentiation is always a straightforward computation is a strong one.
On the other hand, integration is numerically well-defined for piecewise continuous functions, while differentiation isn't and may result in nothing useful. Derivative of a square wave is constant zero with some gaps.
The reason synthesis expertise is not easily recognized as expertise might be because it's hard to evaluate when moving to another system.
Analysis expertise is about knowing the limitations of specific languages, libraries, and frameworks, and this is easy to recognize and evaluate. But synthesis expertise, by its nature, is about 'combining systems within a specific company.' When you change jobs, it's hard to apply that combination to a completely different system.
For example, even if you know why a company's API design and structure were shaped the way they are, that doesn't necessarily mean you can use that knowledge directly at your next company. Maybe that's why.
Yes, this is documented in Bloom's Taxonomy for learning/education. Creating is the highest level of understanding.
Remember – Recall facts, definitions, formulas (memorization).
Understand – Explain ideas in your own words.
Apply – Use knowledge to solve problems or perform tasks.
Analyze – Break information into parts and identify relationships.
Evaluate – Judge, critique, or justify decisions.
Create – Synthesize ideas to produce something new.
The analysis and synthesis approaches to understanding systems have respectively been the driving forces for two major breakthroughs in 20th century physics: reductionist and emergent phenomena. Reductionism aims to understand a system by reducing it to component parts and assuming that the composition is simple. This attitude drives particle physics. On the other hand, one of the most important themes from condensed matter physics has been how "more is different" and collective phenomena can induce emergent behaviour which is very different from the behaviour of the constituent parts. In this perspective, the precise constituent parts don't really matter too much -- many substrates which like completely different to analysis can end up looking very similar in synthesis. This is the principle behind universality classes in critical phenomena. This patter of thought should also be familiar to folks who advocate for a "systems perspective".
In the language of differential vs integral calculus, you can have perfectly well behaved and physical functions whose derivatives can completely miss the global behaviour of the function i.e. smooth but not analytic eg exp(-1/x) at x=0. Funnily enough, this is exactly the form of the action taken by quantum mechanics and an argument for how the classical limit irrecoverably ignores the physics of quantum systems.
I loved reading this article. It was reasonably short to not make the reader lose interest. It jumped around in different domains to make a core point at the end about SREs managing complex systems. This gets even more difficult with the rate of change each of those systems has with coding agents. I don't know if I agree with the terms "synthesis" and "analysis" as the equivalent of global and local respectively, but it was a great read.
Same.
I thought it was interesting that the author mentioned using AI for the integral - they clearly didn't use it for the prose which is exceptional!
I miss reading high quality stuff like this.
Agree
This post brought to mind a post by Bret Victor titled Up and Down the Ladder of Abstraction [1] where he frames working on and understanding problems at different layers of focus as providing insight that isn’t otherwise so apparent. It’s one of those posts that immediately resonated and that I’ve tried to internalize and adopt when approaching problems, especially those in new domains.
[1] https://worrydream.com/LadderOfAbstraction/
If my understanding of Kant is correct, then all analysis (i.e. taking things apart) is synthesis (i.e. putting things together) in reverse, and therefore the latter is a necessary prerequisite of the former. According to him, our minds very first brush with the real world (whatever that might be) is in the form of the "manifold" of raw sensory signals sans any form whatsoever (not even spatial or temporal arrangement), and the only way we can make sense of that manifold of signals or data is to reduce them through a process of combination by commonalities and separation by lack thereof. All objects, categories and concepts ultimately spring from this process of synthesis, some of it involuntary (e.g. the arrangement of certain inputs into three dimensional space) and some of it volitional (e.g. the positing of concepts and categories). Analysis is taking these constructs apart, much like one takes apart a mechanical watch. Except that the exercise tells us more about how our brain puts things together than what the thing actually is.
> Synthesis is harder than analysis
Taking this statement at face value, it means something in computer science: computing an answer (synthesis) is currently believed to be harder than checking an answer (analysis).
The simplest example to illustrate the claim is that factoring a number is harder than multiplying the two factors to check that it equals the original number, or even to decide whether the original is prime or composite (but without yielding the factors).
This also cuts to the heart of what NP means - it means that the answer to a yes/no problem about binary strings can be checked in polynomial time. It doesn't give a recipe for how to generate the answer, but it is implied that finding an answer can take up to exponential time and no more.
>incident response is one area where we are frequently confronted with synthesis problems: we have to understand how the pieces normally fit together in order to make sense of what is currently going wrong.
I'm not sure I can accept without further clarification equating that with synthesis. Normally in incident response, you start with a full system where in principle any root cause is possible and gradually reduce scope until finding the solution.
For me integration could be used for say taking three adhoc solutions and designing a general one that fits all, or merging two different services after a company merge. But anything debugging is very clearly analytic.
> Unfortunately, in our industry we haven’t recognized building up synthesis expertise as a first-class thing.
Good shout out. Same as with integral calc though, a bag of common tricks can get you pretty far. Would make for a good training regimen.
I get the point of the author about differentiation and integration. But, I did not follow completely how he connected it to analysis and synthesis. I mean, there isn't necessarily a one-to-one mapping between analysis and differentiation and synthesis and integration, right?
Maybe something like: When you do derivation you analyze how the function behaves at a given point. Whenever you analyze something you analyze a given thing, not all things. When you do integration you look at a function and then produce something that describes the function on every point. You come up with a closed integral-formula for the shole function. You need to synthesize something that describes the function everywhere. But when you derivate the dericate, you also produce a closed function that describes the function everywhere. Oops, I think I got lost here too.
AI has made me appreciate synthesis much more.
Understanding existing code or generating possible solutions is getting easier. The hard part is still deciding what to build, what not to build, and how independent ideas fit together into something coherent.
In my experience, AI reduced the cost of implementation far more than it reduced the cost of synthesis.
As someone who's always loved synthesizing ideas and having lightbulb moments, I find the headline flattering. I wonder, though, if there's any more rigorous and general analysis (hah!) of the complexity of these two modes of thought.
EDIT: Fable 5 turned up some relevant references:
[1] Richardson, D. (1968). "Some Undecidable Problems Involving Elementary Functions of a Real Variable." Journal of Symbolic Logic, 33(4). — Differentiation is a simple recursive algorithm; deciding integrability in elementary terms is undecidable in general.
[2] Risch, R. H. (1969). "The Problem of Integration in Finite Terms." Transactions of the American Mathematical Society, 139. — The partial recovery: a (semi-)decision procedure for a restricted class.
[3] Guilford, J. P. (1967). The Nature of Human Intelligence. McGraw-Hill. — The divergent vs. convergent thinking distinction, the classic psychometric cousin of synthesis vs. analysis.
[4] Anderson, L. W., & Krathwohl, D. R. (Eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing (revision of Bloom's taxonomy). Longman. — Moved "Create" (synthesis) to the top of the cognitive hierarchy, above "Analyze."
[5] Aaronson, S. (2011). "Why Philosophers Should Care About Computational Complexity." arXiv:1108.1791. — Argues complexity asymmetries (verification vs. generation, P vs. NP) bear directly on questions about cognition.
The point I was waiting for but never came, so I'll make it here, is: Will LLMs be able to synthesize? Sure they can learn calculi. Lambda, differential, whatnot. But will Claude eventually be able to come up with a genuinely new formalism? A new calculus? Sth that allows us to do things we couldn't do before?
P vs NP all over again.
> (Note: I asked AI for the integral of the Gaussian, I hope it got it right!)
It seems like malpractice to not even check this.
It's an illustration for a blog post on a largely different subject, it really doesn't matter.
Who would be looking up the integral of one of the most common functions in applied math in a random philosophical article aimed mostly at SREs?
All you have to do is open the wikipedia page on erf and it's right there.
I don’t think the word applies to a blog post which its self-described as “ramblings”. The formula’s there to look scary and illustrate a point, nobody’s using it to integrate a Gaussian.
Though some people will train LLMs on it and if it's wrong then it will bias the training outcome towards that incorrectness.
But it's not the purpose of a blog post to ensure LLM training data has high quality.
> And it turns out that it’s quite straightforward to calculate a derivative, no matter what type of function it is.
I get the author's point but this is not completely true; there exist functions that are not differentiable at certain places (e.g. ideal square waves) and others that are not differentiable anywhere (e.g. Weierstrass functions).
https://en.wikipedia.org/wiki/Weierstrass_function
It's important to know that (in the usual setting of analysis) not every function is everywhere (or even anywhere) differentiable, but this is more orthogonal to the author's point than opposed to it. A square wave is piecewise differentiable and you can compute a piecewise derivative. The Weierstrass function is defined by an infinite series, and you can compute its derivative term-by-term by the usual rules and check that the result does not converge; it is indeed straightforward to calculate its nonexistent derivative, and this is what Weierstrass did!
In general, to even ask what it means to compute a derivative we need to specify some input language which describes functions in finite terms; we are necessarily in the world of constructions rather than (say) arbitrary set-theoretical maps between infinite sets. With this in mind, the claim that differentiation is always a straightforward computation is a strong one.
Fair point. The mathematical process of differentiation is always straightforward even if in some cases one ends up with e.g. /0 singularities.
With integration, there's often no closed-form process at all, as the author points out.
On the other hand, integration is numerically well-defined for piecewise continuous functions, while differentiation isn't and may result in nothing useful. Derivative of a square wave is constant zero with some gaps.
FWIW in numerical analysis stable differentiation is harder than integration.
But it's quite straight forward to identify that too.
Not always. As an exercise, try to find a continuous real function which is differentiable if and only if the Riemann hypothesis is true!
The reason synthesis expertise is not easily recognized as expertise might be because it's hard to evaluate when moving to another system.
Analysis expertise is about knowing the limitations of specific languages, libraries, and frameworks, and this is easy to recognize and evaluate. But synthesis expertise, by its nature, is about 'combining systems within a specific company.' When you change jobs, it's hard to apply that combination to a completely different system.
For example, even if you know why a company's API design and structure were shaped the way they are, that doesn't necessarily mean you can use that knowledge directly at your next company. Maybe that's why.
> However, when someone says “calculus” without modification […] there’s no ambiguity about which calculus they are referring to
Might be differentiation and integration, might be dental plaque. ;)
Yes, this is documented in Bloom's Taxonomy for learning/education. Creating is the highest level of understanding.