News
Entertainment
Science & Technology
Life
Culture & Art
Hobbies
News
Entertainment
Science & Technology
Culture & Art
Hobbies
Pete Warden discusses the importance of captions, highlighting their utility in training speech-to-text AI and advocating for inclusion in more web videos. He announces the launch of MoonshineJS, a tool for developers to add captions easily. While machine-generated captions aren't perfect, they can enhance accessibility for those needing them.
Sharp Beach in Pacifica looking south, on a beautiful Sunday afternoon Not long after I'd first moved to San Francisco, I had a friend from the UK visit. My apartment overlooked the N Judah tram line, and seeing the "Ocean Beach" destination board on the trains made her want to check it out. So one…
Man facing down three tanks in Tiananmen Square, taken by Jeff Widener of the Associated Press. I love the country of China, I made some good friends while collaborating with engineers based in Beijing, but the Chinese government is a repressive regime relying on censorship to control its own population. I'm old enough to remember feeling a rush…
https://www.youtube.com/watch?v=T1ksM9aFlXY I've never been much of a podcast or video creator but recently I've started posting a series of short chats with my friend Ann Spencer on YouTube and it's been a lot of fun. I realized I hadn't mentioned it here, so as they say, please like and subscribe. I've also embedded one of…
I've been writing this blog for nineteen years, and in over 1,100 posts I've never once brought up politics, but I can't ignore what's happening in our country. We're facing such a profound crisis right now in the US that not speaking up at this point would be breaking the oath I took in 2014,…
Guest post by Nat Jeffries, Founding Engineer at Useful Sensors. At Useful Sensors we love using disposable frameworks to deploy on-device transformers. Having built several such frameworks, I realized that, while there are great resources for understanding and training transformer models, there are few guides for deploying them on-device. The following are some lessons I…
I've been using the ONNX Runtime a lot recently, and while it has been a lot of fun, there are a few things I've missed from the TensorFlow Lite world. The biggest (no pun intended) is the lack of tools to shrink the model file size, something that's always been essential in the mobile app…
When I first tried ChatGPT, it blew my mind. Its ability to respond intelligently to almost any prompt I gave it was astonishing, it was obvious to me it was the future. It seemed like we'd finally built the kind of AI we've all seen in the movies. Over time though, one big limitation became…
Can you imagine using a keyboard where it took a key press two seconds to show up on screen? That's the typical latency for most voice interfaces, so it's no wonder they've failed to catch on for most people. Today we're open sourcing Moonshine, a new speech to text model that returns results faster and…
I've long been a fan of Qualcomm's NPUs, and I even collaborated with them to get experimental support for the underlying HVX DSP into TensorFlow back in 2017 (traces remain here). That meant I was very excited when I heard they were bringing those same accelerators to Windows tablets, offering up to 45 trillion ops…
https://www.youtube.com/watch?v=p2ZFIAGo0VQ I'm excited to announce Torre, a new product that translates instantly between Spanish and English. A lot of native English speakers I talk to don't understand why a better approach to translation is needed, since there have been phone apps around for years. The best way I've found to explain is "Can you imagine…
Have you ever wondered why ChatGPT and similar advanced AI systems are known as Large Language Models? What are "language models", even? To answer that, and understand how remarkable the current state of the art is, I need to jump back a few decades. Understanding language has always been a goal for artificial intelligence researchers,…
According to a survey last year, less than 50% of appliances that are internet-capable ever get connected. When I talk to manufacturers, I often hear even worse numbers, sometimes below 30%! Despite many years and billions of dollars of investment into the “Internet of Things”, this lack of adoption makes it clear that even if…
A few months ago I started updating TensorFlow Lite Micro for the Raspberry Pi Pico board, which uses the RP2040 microcontroller. I ran into some baffling bugs that stopped me making progress, but eventually I tracked them down to my poor understanding of the memory layout. Since I had to do a deep dive, I…
Back in 2020 Foone Turing caused a sensation when she showed Doom running on a pregnancy test. For anyone who remembered desktop computers from the 90's, it was amazing to see a disposable device run something that used to take thousands of dollars worth of hardware. It's not a fluke either - calculators, ATMs, fridges,…
I got my drivers license at 17, on the third attempt, but I never owned a car in the UK since I always biked or took public transport to work. When I was 25 I moved to Los Angeles, so I had to become a car owner for the first time. I wasn't looking for…
As many of you know, I'm an old geezer working on a CS PhD at Stanford and part of that involves me taking some classes. The requirements are involved, but this quarter I ended up taking "Hack Lab: Introduction to Cybersecurity". I was initially attracted to it because it focuses on the legal as well…
Imagine asking a box on a pillar at Home Depot “Where are the nails?” and getting directions, your fridge responding with helpful advice when you say “Why is the ice maker broken?”, or your car answering “How do I change the wiper speed?”. I think of these kinds of voice assistants for everyday objects as…
As you might know I'm working on my PhD at Stanford, and one of my favorite parts is taking courses. For this second year I need to follow the new foundation and breadth requirements which in practice means taking a course a quarter, with each course chosen from one of four areas. For the fall…
https://youtu.be/Uf6NJuyVKG0 We all grew up with TV shows, books, and movies that assume everybody can understand each when they speak, even if they're aliens. There are various in-universe explanations for this convenient feature, and most of them involve a technological solution. Today, the Google Translate app is the closest thing we have to this kind…
Photo by Steve Harwood On Friday my long-time colleague Nat asked if we should try and expand our Useful Transformers library into something that could be suitable for a lot more use cases. We worked together on TensorFlow, as did the main author of UT, Manjunath, so he was surprised when I didn't want to…
At Useful Sensors we're focused on building intelligent sensors, ones that use machine learning to take raw data and turn it into actionable insights. Sometimes I run across problems in my own life that don't need advanced algorithms or AI to solve, but are blocked by hardware limitations. A classic one is "Did I leave…
Tomorrow I'll be giving a remote talk at the LBQNN workshop at ICCV. The topic is the history of quantization in machine learning, and while I don't feel qualified to give an authoritative account, I did think it might be interesting to cover the developments I was aware of. I don't know if the talk…
Nvidia is an amazing company that has executed a contrarian vision for decades, and has rightly become one of the most valuable corporations on the planet thanks to its central role in the AI revolution. I want to explain why I believe it's top spot in machine learning is far from secure over the next…
I've been a fan of the RP2040 chip powering the Pico since it was launched, and we're even using them in some upcoming products, but I'd never used one of its most intriguing features, the second core. It's not common to have two cores in a microcontroller, especially a seventy cent Cortex M0, and most…
It's weird to live in a place that so many people have heard of, but so few people know. Silicon Valley is so full of charismatic people spinning whatever stories serve their ends it's hard for voices with fewer ulterior motives to get airtime. Even the opponents of big tech have an incentive to mythologize…
https://youtu.be/9JZlAe_NFlY Video of an AI-controlled lamp There's a lot of hype around AI these days, and it's easy to believe that it's just another tech world fad like the Metaverse or crypto. I think that AI is different though, because the real-world impact doesn't require a leap of faith to imagine. For example, I've had…
I love Brad DeLong's writing, but I did a double take when he recently commented "'A Young Lady's Illustrated Primer' continues to recede into the future". The Primer he's referencing is an electronic book from Neal Stephenson's Diamond Age novel, an AI tutor designed to educate and empower children, answering their questions and shaping their…
Photo by Gopal Vijayaraghavan My startup, Useful Sensors, has all of its money in Silicon Valley Bank. There are a lot of things I worried about as a CEO, but assessing SVB's creditworthiness wasn't one of them. It clearly should have been. I don't have any grand theories about what's happened over the last few…
https://www.youtube.com/watch?v=AZzYZOzvCG8 When I think of a museum with "craft" in its name, I usually imagine an institution focused on the past. San Francisco's Museum of Craft and Design is different. Their mission is to "bring you the work of the hand, mind and heart", and Bull.Miletic's Proxistant Vision exhibition is a wonderful example of how…
If you've heard me on any podcasts recently, you might remember I've been talking about a Gesture Sensor as the follow up to our first Person Sensor module. One frustrating aspect of building hardware solutions is that it's very tough to share prototypes with people, since you usually have to physically send them a device.…
Years ago I used to write regular "Five Short Links" posts but I gave up as my Twitter account became a better place to share updates, notes, and things I found interesting from around the internet. Now that Twitter is Nazi-positive I'm giving up on it as a platform, so I'm going to try going…
Person Sensor from Useful Sensors For years I've wanted to be able to look at a light switch, say "On", and have the lights switch on. This kind of interface sounds simple, so why doesn't it exist? It turns out building one requires solving a lot of tough research and engineering challenges, and even more…
Sparko, the world's first electrical dog, as he looked on arrival at the engineer's club, New York City, on his way to the World's fair, where he will be an attraction at the Westinghouse Building. He walks, barks, wags his tail and sits up to beg. With Sparko, is Elektro, Westinghouse mechanical man. Both are…
I was digging through paperwork today to help complete my PhD admission process, and I stopped short when I saw the academic transcript from my undergraduate years. I was a terrible student! I got 0% on one course, awful scores on many others, and had to do a lot of retakes. It brought back memories…
Photo by Ian Sane Last week I had a question from a colleague about reproducibility in TensorFlow, specifically in the 1.14 era. He wanted to be able to run the same training code multiple times and get exactly the same results, which on the surface doesn't seem like an unreasonable expectation. Machine learning training is…
Photo by the National Park Service I've been working on a new research paper, and a friend gave me the feedback that he was confused by the statement "memory accesses can be accurately predicted at the compilation stage" for machine learning workloads, and that this made them a poor fit for conventional processor architectures with…
One of the most frequent questions I get asked from people exploring machine learning beyond cloud and desktop machines is "What about training?". If you look around at the popular frameworks and use cases of edge ML, most of them seem focused on inference. It isn't obvious why this is the case though, so I…
https://youtu.be/-bRBCVfF-q0 i-FlatCam demo I've finally had the chance to play Cyberpunk 2077 over the last few weekends, and it's an amazing feat of graphics programming, especially with ray-tracing enabled. I've had fun, but I have been struck by how the cyberpunk vision of the future is rooted in the '80s. Even though William Gibson was…
I've spent a lot of time at conferences talking about all the wonderful things that are now possible using machine learning on embedded devices, but as Stacey Higginbotham pointed out at this year's TinyML Summit, despite all the potential there haven't been many shipping applications. My experience is that companies like Google with big ML…