The framework used in the book, malt[0], is currently not GPU-accelerated, but it's being worked on.
Maybe interesting, I used it for a toy implementation of the GPT architecture[1] in about 500 lines.
(I studied with one of the authors, Dr. Daniel Friedman; wasn't super involved here but proofread a late draft and TA'd for a course based off the book.)
I'm a huge fan of project based learning like the approach taken in this book. But I'm not sure if it's a good idea to introduce early stage students to Scheme before Python, or deep learning before calculus.
I studied pure math in college, and we were required to take 2 "Computer Science" classes as part of that program. Mainly memorizing textbook algorithms and data structure implementations in Java. I hated programming for years after that, until during graduate school I came up with a project of my own that organically required knowledge of Matlab and later Python. I loved programming after that.
I hope books like this can help new students avoid the trough of disillusionment that can sometimes happen if you're forced to learn a cool subject (like programming) in a very uncool way.
Personally, I would not recommend this book to a young person interested in deep learning and programming (based on the table of contents). I would probably recommend they first learn calculus and use Python to make plots while doing so. Then read Fleuret's "The Little Book of Deep Learning" and try to implement simple models in PyTorch.
> to introduce early stage students to Scheme before Python, or deep learning before calculus.
This book is part of the classic "Little" series of books starting with the "Little Schemer" which, despite its name and style, is certainly not a novice beginner book.
The later books: "Seasoned Schemer", "Reasoned Schemer", "The Little Typer" and "The Little Prover" are all very advanced books. They share the same style of illustrations and Socratic method, but you will absolutely need to work through them slowly and careful to get value out of them.
The "Little" books are generally targeted to an audience of computer language nerds and pretty much assume you have a solid understanding of programming, familiarity with scheme and the books come from a time when every serious engineer had basic calc knowledge.
These are classics (and I was really impressed with "The Little Learner") but are very serious and challenging texts, that outside the first book, are aimed at advanced readers (and for those readers are true delights).
I think you're overstating the difficulty of Seasoned and Reasoned, both are comfortably undergraduate texts (though Reasoned is a challenging text if it's your first time with that style of programming). We used to teach Seasoned to high school students (advanced high school students, but I'd put them on par with motivated 1st/2nd year college students not more advanced than that). I'd agree with the other three, though, they are advanced texts.
This was just standard procedure at university in 2002. Nothing special. It was me and 200 more people.
Pretty sure all the exercises were lifted or inspired by The Little Schemer even thought it took me years to learn about that book.
Telling people that they need to learn boring prerequisites before they are allowed to learn the next thing is exactly how you kill motivation. This strategy works if you have a fully mapped out curriculum that is planned as a whole and a societal expectation that you go through the entire thing, it doesn't work for independent learning.
For independent learning, motivation is the key bottleneck and your goal is to make people organically motivated in calculus after learning that their calculus knowledge is lacking.
The first programming language one learns is really not important, most skills are transferable. Racket is an excellent programming language for beginners (this book however is not a good book for a complete beginner because it goes really fast over the Racket introduction).
I wouldn't recommend it. My copy is dogeared - but for me as a concise mini-reference for a working statistician who has to work with data science people, but not actually implement anything. It's more of a high-level handwavy "these are the concepts involved".
So your opinion is based on just reading the table of contents? I always find it disconcerting when someone writes a multi-paragraph commentary on a work they didn't actually read or see.
I understand that you're commenting on the approach more than the contents, but you're pretty dismissive of it without actually reading the details of how they went about things.
You're not quite judging a book by its cover, but you're not that far beyond that.
> you're pretty dismissive of it without actually reading the details of how they went about things...
I wasn't really trying to be dismissive (other than saying that I personally would not recommend it to a young person interested in programming and deep learning). I was mostly trying to start a discussion about the best way to teach/learn this subject. I hoped to attract more knowledgeable commenters (such as yourself). A day later I still stand by my personal opinion that it's probably best to learn the mathematics first, and to use the lingua franca of the domain.
I'd like to add that, from my perspective, this pseudo-critique was a very small part of my comment. I was mostly trying to say "It's very important and difficult to keep early students' interest. Bravo on the novel approach taken in the book. It's much better than what I had as a college student." That might not have been clear in my comment.
> You're not quite judging a book by its cover, but you're not that far beyond that.
Fair. But there wasn't anything else to read in the submission and I was trying to start a curious conversation. Despite my good intentions it was a bad comment.
The framework used in the book, malt[0], is currently not GPU-accelerated, but it's being worked on.
Maybe interesting, I used it for a toy implementation of the GPT architecture[1] in about 500 lines.
(I studied with one of the authors, Dr. Daniel Friedman; wasn't super involved here but proofread a late draft and TA'd for a course based off the book.)
[0]: https://github.com/themetaschemer/malt
[1]: https://github.com/sporkl/malt-transformer
Related. Others?
The Little Learner: A Straight Line to Deep Learning - https://news.ycombinator.com/item?id=34810332 - Feb 2023 (96 comments)
[dead]
From 2023,
"Pub date: February 21, 2023"
Yes should be tagged (2023) imo
I'm a huge fan of project based learning like the approach taken in this book. But I'm not sure if it's a good idea to introduce early stage students to Scheme before Python, or deep learning before calculus.
I studied pure math in college, and we were required to take 2 "Computer Science" classes as part of that program. Mainly memorizing textbook algorithms and data structure implementations in Java. I hated programming for years after that, until during graduate school I came up with a project of my own that organically required knowledge of Matlab and later Python. I loved programming after that.
I hope books like this can help new students avoid the trough of disillusionment that can sometimes happen if you're forced to learn a cool subject (like programming) in a very uncool way.
Personally, I would not recommend this book to a young person interested in deep learning and programming (based on the table of contents). I would probably recommend they first learn calculus and use Python to make plots while doing so. Then read Fleuret's "The Little Book of Deep Learning" and try to implement simple models in PyTorch.
> to introduce early stage students to Scheme before Python, or deep learning before calculus.
This book is part of the classic "Little" series of books starting with the "Little Schemer" which, despite its name and style, is certainly not a novice beginner book.
The later books: "Seasoned Schemer", "Reasoned Schemer", "The Little Typer" and "The Little Prover" are all very advanced books. They share the same style of illustrations and Socratic method, but you will absolutely need to work through them slowly and careful to get value out of them.
The "Little" books are generally targeted to an audience of computer language nerds and pretty much assume you have a solid understanding of programming, familiarity with scheme and the books come from a time when every serious engineer had basic calc knowledge.
These are classics (and I was really impressed with "The Little Learner") but are very serious and challenging texts, that outside the first book, are aimed at advanced readers (and for those readers are true delights).
I think you're overstating the difficulty of Seasoned and Reasoned, both are comfortably undergraduate texts (though Reasoned is a challenging text if it's your first time with that style of programming). We used to teach Seasoned to high school students (advanced high school students, but I'd put them on par with motivated 1st/2nd year college students not more advanced than that). I'd agree with the other three, though, they are advanced texts.
That's the video of my son to prove that Scheme can be a first programming language.
https://www.youtube.com/watch?v=LXhsutNKhec
This is how he learned Scheme.
1. Typing 2. Vim through https://vim-adventures.com/ 3. Little Schemer https://a.co/d/0343Ez4v
Don't assume that they would need more things.
This is child abuse. What sort of monster uses vim for s-expression based languages? Everyone knows Emacs is the only sane way to work with them.
I mostly learned with pen and paper and tracing in my mind. We were 18 though
Pen and paper? Excuse me, but real programmers use butterflies.
This was just standard procedure at university in 2002. Nothing special. It was me and 200 more people. Pretty sure all the exercises were lifted or inspired by The Little Schemer even thought it took me years to learn about that book.
I suppose butterflies would do for volatile storage but you really need long term memory, such as twig and clay tablet.
What was good enough for the Sumerian’s is good enough for me.
This is also pretty good
https://www.sscardapane.it/alice-book/
Telling people that they need to learn boring prerequisites before they are allowed to learn the next thing is exactly how you kill motivation. This strategy works if you have a fully mapped out curriculum that is planned as a whole and a societal expectation that you go through the entire thing, it doesn't work for independent learning.
For independent learning, motivation is the key bottleneck and your goal is to make people organically motivated in calculus after learning that their calculus knowledge is lacking.
The first programming language one learns is really not important, most skills are transferable. Racket is an excellent programming language for beginners (this book however is not a good book for a complete beginner because it goes really fast over the Racket introduction).
Python has so many ways of doing things it's a distraction, doubly so when you add all the special sauce for tensor manipulation.
Scheme by comparison has only one way of doing things and gets out of your way.
> "The Little Book of Deep Learning"
I wouldn't recommend it. My copy is dogeared - but for me as a concise mini-reference for a working statistician who has to work with data science people, but not actually implement anything. It's more of a high-level handwavy "these are the concepts involved".
There are quite a lot of precedents for teaching Scheme as a first language.
Java is not a great choice and one I can see would be offputting.
N=1, but I learned scheme first. It was great and I'm doing fine. First scheme then C. Back-to-back.
> (based on the table of contents)
So your opinion is based on just reading the table of contents? I always find it disconcerting when someone writes a multi-paragraph commentary on a work they didn't actually read or see.
I understand that you're commenting on the approach more than the contents, but you're pretty dismissive of it without actually reading the details of how they went about things.
You're not quite judging a book by its cover, but you're not that far beyond that.
> you're pretty dismissive of it without actually reading the details of how they went about things...
I wasn't really trying to be dismissive (other than saying that I personally would not recommend it to a young person interested in programming and deep learning). I was mostly trying to start a discussion about the best way to teach/learn this subject. I hoped to attract more knowledgeable commenters (such as yourself). A day later I still stand by my personal opinion that it's probably best to learn the mathematics first, and to use the lingua franca of the domain.
I'd like to add that, from my perspective, this pseudo-critique was a very small part of my comment. I was mostly trying to say "It's very important and difficult to keep early students' interest. Bravo on the novel approach taken in the book. It's much better than what I had as a college student." That might not have been clear in my comment.
> You're not quite judging a book by its cover, but you're not that far beyond that.
Fair. But there wasn't anything else to read in the submission and I was trying to start a curious conversation. Despite my good intentions it was a bad comment.