I didn't scour the text but one thing I didn't see as an explanatory consideration in the paper is that the larger companies they talk about having larger adoption are also much more likely to have reasonably robust controls (and consequences) that prevent mixing personal/corporate use of AI tooling. That's my world.
On the other hand my girlfriend is a teacher and for the past year ChatGPT has allowed her to augment her lesson plans with slides/games/etc that have had a very positive effect on student engagement. She also uses it to help with analysis of testing data against her lesson plans and state standards to find places to shore things up as the year progresses. She runs the english language learning department in her school and also uses it to help ensure that there is representation in her materials for the huge mix of cultural backgrounds of her students.
It's truly helped her immensely and it's all been on her personal account. The district has no program or budget to guide or support teachers and admin on using these tools in education.
Schools for the poorer segments of society keep adding more and more technology, eroding the analytical skills of their students and staff. But the shiny things keep the poorer parents and mediocre teachers happy while keeping down cost per lesson per child. This keeps the administrators happy.
Meanwhile ultra-expensive elite private schools for the children of the Zuckerbergs, Al Sauds, Al Nahyans, where the teachers are Ivy and Oxbridge graduates emphasise tradition, classical studies and analytical skills while relegating technology to a minor role
As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...
This was more than a decade ago when we were looking into emersion schools for our daughter. We went to several private schools first and all three were emphasizing ipads as their primary tool for teaching language. The public school our daughter was set to attend had an open house promoting its long running emersion programs.
Wow. Our public school had native speakers and no ipads. There was a major focus on true emersion. From minute one, they were spoken to in the native language. The teachers were incredible. In the first six months, my daughter was speaking in full sentences and her teachers were praising her pronunciation of certain words.
The real eye opener came at a local park. My daughter was playing with a boy, speaking to him and his mother. I came over to join the fun and the mother started speaking to me in Spanish. I was like, "Uh, I don't speak Spanish." She looked at me confused and said, "Your daughter, she was speaking fluently to both myself and my son in Spanish."
It was a pretty jaw dropping moment. One, to think we were blessed to have such a great emersion program at our public school and secondly to see my daughter pick up a language so effortlessly. It really reinforced my idea that its all about the quality of education, not necessarily how much you're paying for it.
This is incredible to hear! My gf has had similar stories about kids coming into her program with zero English, often with parents that have little to no English, and within a short amount of time becoming conversationally fluent and translating for their mom and dad.
> But the shiny things keep the poorer parents and mediocre teachers happy while keeping down cost per lesson per child.
Baltimore spends more per student than Switzerland.
> As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...
Education for more affluent people is often cheaper per student. The benefit richer students have is that, on average, their parents are smarter, more conscientious, etc.
The Rotchschilds, the Rockefellers, and the Du Ponts would have done very well in life just as well had they went to public schools, while the alumni of Baltimore public school would not become Rotschilds and Rockefellers and Zuckerbergs had they gone to Pillips Exeter.
> As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...
The rich and powerful pay "fuck you" money for their kids to get the best pre-college educations possible. It takes an incredible amount of willpower for a world-class teacher to say no to that type of money over even high-paying private schools.
Prior to her current role my gf spent about a decade as the head of a very elite private elementary school (she's been in education for 35 years now). Certainly ten figure families represented in the student body with names you would recognize as such. The school had access to all of the resources they could ask for and they did very well on paper. However, the attitude of the students was qualitatively different (and in my gf's opinion much worse) than those of her current students.
It's fun to imagine a collection of rock star educators huddled around every child just constantly feeding their intellectual curiosity, but the reality is most of those kids just aren't hungry. They live lives of comfortable inconsequentiality which, on average, seems to rob something from human curiosity and drive.
Keeping down cost per child? What are you talking about? The cost per child has skyrocketed in the US over last 20 years, particularly among the worst-performing schools. Baltimore spends more per child than Mountain View. NYC spends more per child than Palo Alto.
There is a certain view, popular among many people, that poor performing schools perform badly because they are underfunded. It has been vigorously spread by politicians, teachers, and activists. Well, we ran the experiment. With enormous federal and state spending, we managed to mostly equalize the spending, and, in many cases, fund the badly performing schools more than the good ones. The results of the experiment are very clear: additional funding, by and large, does not *cause* significantly improved educational outcomes. Any observed correlation between school funding and outcomes is a result of who lives in the area, and who attends the school. The casual arrow does not go from funding to outcomes, but from the school population to both funding, and, in parallel, to outcomes.
Increased funding did improve educational outcomes. It just didn't improve it enough to be considered worthwhile.
The fact is that educational outcomes are caused by a large number of variables. Areas with (historically) poorer schools have less competent (or more corrupt) administrations so money doesn't go as far. Even with good leadership at the top (superintendent or public school committee), if the source of that money is not something you can rely on consistently enough to pay salaries with then you are not going to see significant changes.
Pre-K education is also critical to determining later school performance, as knowing the alphabet and numbers are practically assumed by kindergarten curriculums as they are covered far too far to be more than a review. But we have not tried that yet, so we'll see if that pans out...
> my girlfriend is a teacher and for the past year ChatGPT has allowed her to augment her lesson plans with slides/games/etc that have had a very positive effect on student engagement.
My wife, who is also a teacher, does the same thing. I think this is an underrated use of AI right now. There's also been a new wave of edtech companies focusing on AI as the teacher, which I'm skeptical will reach product market fit. I think AI as the teacher assistant could be interesting though.
IMO, in everything, there will be a small cohort of incredibly motivated and dedicated to always giving their all for their job, which is also their passion in life. Then there are the people who at one point had that passion^, and are now doing the work to cash in a paycheck, and will take shortcuts where possible to replace the effort they would be putting in.
^ not that this is always something to blame people for. A passion to do something you love doesn't typically translate into doing it so that you enable your employer to profit.
I think a majority of my friends are teachers and this could not be further opposite their sentiments. They feel strongly that it’s dishonest, nearly akin to plagiarism to use AI output in lesson plans. They don’t use it for grading or data analysis b/c of privacy concerns. They mostly complain that the kids are using it for homework, which makes it hard to grade both because it’s too uniform and because there’s no good faith effort going into the work which makes reading it unrewarding. And the consensus is that test scores are down b/c they’re offloading normal course work to AI. It’s basically the bane of their existence.
The idea that it’s actually better for diversity feels like a wild reach, almost a canned talking point.
Would it kill these OpenAI guys to learn how to write a white paper? Google's DeepMind did better with their AGI to ASI, though I have qualms on their execution.
Whenever I see these Frontier Labs "publish a paper" on ArXiv.org the phrase Esse Quam Videri strides past my mind—is this genuine progress or merely a marketing metric for their stakeholders?
When I say “how” I mean like how u/glouwbug mentioned: Formatting. But more so Structure.
When I read artifacts by Frontier Labs I notice a trend towards deteriorating lexicographic architecture. Like how a textbook or larger tomb is constructed for the reader to navigate.
Instead, now I see continuous monotonic text. It’s as if the author expects us to chuck the artifact into an LLM in order for it to inform us of meaning.
I feel like this is a common format for social sciences or economics papers, e.g. most of the papers I've seen on ssrn.com have a similar structure.
To be sure, I always did find those a bit hard to read (they seem to never have evolved past the stage where papers were read on, y'know, paper instead of screens) but I think this paper is simply keeping with the conventions for studies on this subject matter.
The figures down 80 scrolls towards the bottom is what bothered me also.
Most of the time with studies, the figures are where you should go first. It's not uncommon to find cases where the text makes claims that the numbers don't support. It also, pre AI era, took at least some extra effort to make data visualizations, so the authors would focus on visualizing the most useful/impactful/interesting data sets.
It did make me ask why I wanted to see them while reading, and if there was any value in just reading the text without the images filling in the blanks (and possibly biasing what I was focusing on).
Concluded that I still want to see the images right around the time the are referenced.
Purely anecdotally, over the last few weeks I've noticed a significant uptick in articles trying to convince me how awesome AI is doing. And that's from a pretty high baseline of articles that were already telling me how I should really appreciate how awesome AI is.
It's almost as though there's some worry that the wheels are going to come off before the right people are able to cash out.
isnt it how marketing is done these days? people/companies buy views, clicks, articles, posts on social media saying how great something/someone is or how its changed their life etc etc.
it really becomes hard to separate the wheat/chaff for alot of these things...
They draw a random sample of matched adopters but use the full Compustat population of US companies as non-adopters and don't disclose the sampling rate, which is statistically risky.
Also doesn't account for pilots. If a Enterprise customer has 10 licenses for a pilot, they could be in there.
This reads like some crypto or NFT commodity white paper, i.e. like an ad that says you need to use this now, and more, so that you'll figure out what it's good for.
A difference is that in those white papers they usually argue that their solution is more perfect in some sense than some more adopted alternative, here they instead claim that their commercial offering is "general purpose".
Conclusion: no measureable ROI. In fact, the enterprises have no idea where or how to begin measuring (we don't either tbh). We are still early. Keep buying our product, so we can find out, together.
I didn't scour the text but one thing I didn't see as an explanatory consideration in the paper is that the larger companies they talk about having larger adoption are also much more likely to have reasonably robust controls (and consequences) that prevent mixing personal/corporate use of AI tooling. That's my world.
On the other hand my girlfriend is a teacher and for the past year ChatGPT has allowed her to augment her lesson plans with slides/games/etc that have had a very positive effect on student engagement. She also uses it to help with analysis of testing data against her lesson plans and state standards to find places to shore things up as the year progresses. She runs the english language learning department in her school and also uses it to help ensure that there is representation in her materials for the huge mix of cultural backgrounds of her students.
It's truly helped her immensely and it's all been on her personal account. The district has no program or budget to guide or support teachers and admin on using these tools in education.
Such a shame.
Schools for the poorer segments of society keep adding more and more technology, eroding the analytical skills of their students and staff. But the shiny things keep the poorer parents and mediocre teachers happy while keeping down cost per lesson per child. This keeps the administrators happy.
Meanwhile ultra-expensive elite private schools for the children of the Zuckerbergs, Al Sauds, Al Nahyans, where the teachers are Ivy and Oxbridge graduates emphasise tradition, classical studies and analytical skills while relegating technology to a minor role
As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...
This was more than a decade ago when we were looking into emersion schools for our daughter. We went to several private schools first and all three were emphasizing ipads as their primary tool for teaching language. The public school our daughter was set to attend had an open house promoting its long running emersion programs.
Wow. Our public school had native speakers and no ipads. There was a major focus on true emersion. From minute one, they were spoken to in the native language. The teachers were incredible. In the first six months, my daughter was speaking in full sentences and her teachers were praising her pronunciation of certain words.
The real eye opener came at a local park. My daughter was playing with a boy, speaking to him and his mother. I came over to join the fun and the mother started speaking to me in Spanish. I was like, "Uh, I don't speak Spanish." She looked at me confused and said, "Your daughter, she was speaking fluently to both myself and my son in Spanish."
It was a pretty jaw dropping moment. One, to think we were blessed to have such a great emersion program at our public school and secondly to see my daughter pick up a language so effortlessly. It really reinforced my idea that its all about the quality of education, not necessarily how much you're paying for it.
This is incredible to hear! My gf has had similar stories about kids coming into her program with zero English, often with parents that have little to no English, and within a short amount of time becoming conversationally fluent and translating for their mom and dad.
It's pretty amazing.
I wonder if the word you’re looking for is “immersion” — to be immersed in something — rather than emerging from .. knowledgelessness?
> But the shiny things keep the poorer parents and mediocre teachers happy while keeping down cost per lesson per child.
Baltimore spends more per student than Switzerland.
> As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...
Education for more affluent people is often cheaper per student. The benefit richer students have is that, on average, their parents are smarter, more conscientious, etc.
Baltimore does not spend more per student than Institut Le Rosey, the school of kings... (alumni include the Rothschilds, Rockefellers, the du Ponts)
The fees are well over USD $150,000 a year.
Philips Exeter (Zuckerberg's school ) is a bargain by comparison at $80,000 s year .
These schools emphasis clear and analytical thinking and traditional education.
In contrast the unfortunate children of the poor are subjected to everyone technology and education trend of the moment...
Education in public schools has turned into AI slop being generated by poorly educated teachers.
The Rotchschilds, the Rockefellers, and the Du Ponts would have done very well in life just as well had they went to public schools, while the alumni of Baltimore public school would not become Rotschilds and Rockefellers and Zuckerbergs had they gone to Pillips Exeter.
That doesn’t mean their schools aren’t better. Money isn’t the only reason having a thoughtful, rigorous and well-constructed education is important.
Ah, cause they inherited a ton of cash ya?
Are you defining "poor" to mean everyone who doesn't send their kids to Philips Exeter?
> As always the children of the rich and powerful get bespoke education... You could call it elegant education for a more civilized age...
The rich and powerful pay "fuck you" money for their kids to get the best pre-college educations possible. It takes an incredible amount of willpower for a world-class teacher to say no to that type of money over even high-paying private schools.
Eh.
Prior to her current role my gf spent about a decade as the head of a very elite private elementary school (she's been in education for 35 years now). Certainly ten figure families represented in the student body with names you would recognize as such. The school had access to all of the resources they could ask for and they did very well on paper. However, the attitude of the students was qualitatively different (and in my gf's opinion much worse) than those of her current students.
It's fun to imagine a collection of rock star educators huddled around every child just constantly feeding their intellectual curiosity, but the reality is most of those kids just aren't hungry. They live lives of comfortable inconsequentiality which, on average, seems to rob something from human curiosity and drive.
Keeping down cost per child? What are you talking about? The cost per child has skyrocketed in the US over last 20 years, particularly among the worst-performing schools. Baltimore spends more per child than Mountain View. NYC spends more per child than Palo Alto.
There is a certain view, popular among many people, that poor performing schools perform badly because they are underfunded. It has been vigorously spread by politicians, teachers, and activists. Well, we ran the experiment. With enormous federal and state spending, we managed to mostly equalize the spending, and, in many cases, fund the badly performing schools more than the good ones. The results of the experiment are very clear: additional funding, by and large, does not *cause* significantly improved educational outcomes. Any observed correlation between school funding and outcomes is a result of who lives in the area, and who attends the school. The casual arrow does not go from funding to outcomes, but from the school population to both funding, and, in parallel, to outcomes.
Increased funding did improve educational outcomes. It just didn't improve it enough to be considered worthwhile.
The fact is that educational outcomes are caused by a large number of variables. Areas with (historically) poorer schools have less competent (or more corrupt) administrations so money doesn't go as far. Even with good leadership at the top (superintendent or public school committee), if the source of that money is not something you can rely on consistently enough to pay salaries with then you are not going to see significant changes.
Pre-K education is also critical to determining later school performance, as knowing the alphabet and numbers are practically assumed by kindergarten curriculums as they are covered far too far to be more than a review. But we have not tried that yet, so we'll see if that pans out...
> my girlfriend is a teacher and for the past year ChatGPT has allowed her to augment her lesson plans with slides/games/etc that have had a very positive effect on student engagement.
My wife, who is also a teacher, does the same thing. I think this is an underrated use of AI right now. There's also been a new wave of edtech companies focusing on AI as the teacher, which I'm skeptical will reach product market fit. I think AI as the teacher assistant could be interesting though.
IMO, in everything, there will be a small cohort of incredibly motivated and dedicated to always giving their all for their job, which is also their passion in life. Then there are the people who at one point had that passion^, and are now doing the work to cash in a paycheck, and will take shortcuts where possible to replace the effort they would be putting in.
^ not that this is always something to blame people for. A passion to do something you love doesn't typically translate into doing it so that you enable your employer to profit.
I think a majority of my friends are teachers and this could not be further opposite their sentiments. They feel strongly that it’s dishonest, nearly akin to plagiarism to use AI output in lesson plans. They don’t use it for grading or data analysis b/c of privacy concerns. They mostly complain that the kids are using it for homework, which makes it hard to grade both because it’s too uniform and because there’s no good faith effort going into the work which makes reading it unrewarding. And the consensus is that test scores are down b/c they’re offloading normal course work to AI. It’s basically the bane of their existence.
The idea that it’s actually better for diversity feels like a wild reach, almost a canned talking point.
Lol ok tons of teachers use LLMs to make material. The plural of anecdote is not data, but in this case it's close.
> tons of teachers use LLMs to make material
How do you say this with such confidence and then, in the very next sentence, admit that all you have is anecdata which you don't even bother to cite?
>She also uses it to help with analysis of testing data
>on her personal account
With all the PII stripped, I hope?
OpenAI is PII compliant so it's ok
Would it kill these OpenAI guys to learn how to write a white paper? Google's DeepMind did better with their AGI to ASI, though I have qualms on their execution.
Whenever I see these Frontier Labs "publish a paper" on ArXiv.org the phrase Esse Quam Videri strides past my mind—is this genuine progress or merely a marketing metric for their stakeholders?
you are complaining about a problem with this paper without explaining what it is.
When I say “how” I mean like how u/glouwbug mentioned: Formatting. But more so Structure.
When I read artifacts by Frontier Labs I notice a trend towards deteriorating lexicographic architecture. Like how a textbook or larger tomb is constructed for the reader to navigate.
Instead, now I see continuous monotonic text. It’s as if the author expects us to chuck the artifact into an LLM in order for it to inform us of meaning.
I feel like this is a common format for social sciences or economics papers, e.g. most of the papers I've seen on ssrn.com have a similar structure.
To be sure, I always did find those a bit hard to read (they seem to never have evolved past the stage where papers were read on, y'know, paper instead of screens) but I think this paper is simply keeping with the conventions for studies on this subject matter.
Single column PDF. This is the frontier.
I find single column much easier to read than dual. Though the line spacing is too much in this concrete example.
Granted I didn't comb over it but to me the big issue was that it was just a big data dump without any material attempt at meaningful analysis.
That and all the figures were at the bottom which was kind of annoying.
The figures down 80 scrolls towards the bottom is what bothered me also.
Most of the time with studies, the figures are where you should go first. It's not uncommon to find cases where the text makes claims that the numbers don't support. It also, pre AI era, took at least some extra effort to make data visualizations, so the authors would focus on visualizing the most useful/impactful/interesting data sets.
It did make me ask why I wanted to see them while reading, and if there was any value in just reading the text without the images filling in the blanks (and possibly biasing what I was focusing on).
Concluded that I still want to see the images right around the time the are referenced.
this paper sucks. It was written by capital E Economists and has some very bad figures that are all hidden at the bottom.
Likely just formatting
Making valid complaints would require actually reading it.
This is a typical working paper (preprint) not white paper, for its field. You can wait for the journal publication.
"badly". There, fixed it for you.
Seriously, the best blog post on this subject has to be Ludicity: https://ludic.mataroa.blog/blog/ai-mania-is-eviscerating-glo...
Purely anecdotally, over the last few weeks I've noticed a significant uptick in articles trying to convince me how awesome AI is doing. And that's from a pretty high baseline of articles that were already telling me how I should really appreciate how awesome AI is.
It's almost as though there's some worry that the wheels are going to come off before the right people are able to cash out.
isnt it how marketing is done these days? people/companies buy views, clicks, articles, posts on social media saying how great something/someone is or how its changed their life etc etc.
it really becomes hard to separate the wheat/chaff for alot of these things...
They draw a random sample of matched adopters but use the full Compustat population of US companies as non-adopters and don't disclose the sampling rate, which is statistically risky.
Also doesn't account for pilots. If a Enterprise customer has 10 licenses for a pilot, they could be in there.
This reads like some crypto or NFT commodity white paper, i.e. like an ad that says you need to use this now, and more, so that you'll figure out what it's good for.
A difference is that in those white papers they usually argue that their solution is more perfect in some sense than some more adopted alternative, here they instead claim that their commercial offering is "general purpose".
Question: "How"
Answer: Yes.
Conclusion: no measureable ROI. In fact, the enterprises have no idea where or how to begin measuring (we don't either tbh). We are still early. Keep buying our product, so we can find out, together.
69 pages long.
The main text is only 23 pages double-spaced. The remaining manuscript comprises images, tables, and references. It's not that long really.
the rest of the 'supporting' material adds little.
Blog post: https://openai.com/index/how-enterprises-put-ai-to-work/ (https://news.ycombinator.com/item?id=49281467)