Thoughts on Tech and Things

Latest Thoughts

1. Apple Wanderlust Event Recap 2024

   Did you skip the Apple Wanderlust event on Tuesday? No worries, I’ve got you covered! Join me as I sit down with our Director of Mobile Engineering, Rob Koch, to unpack all the exciting announcements. From the new Apple Watch to Beta testing iOS 17, we’ve got it all covered. Grab a drink, and let’s dive in!

2. Apple’s Walled Garden

   Austin Carr has an interesting piece on leaving Apple’s walled garden. 

   “Over the past few years, though, it all started to feel claustrophobic. I was only a little annoyed when Siri crept into my apps and search queries. Only a little frustrated that iMessage didn’t allow for modern chatting on non-Apple PCs and phones. Only a little miffed that I couldn’t choose Google Maps as my default navigator or set up an Amazon Echo as easily as Apple’s own HomePod speaker. But these things began to add up, as did the $120 I was spending every year to store my photos on iCloud.”

   I’m a pretty happy Apple fanboy, but I spend a lot of time experimenting with and using many different types of hardware. Amazon Echo’s release was one of the first moments I felt a bit claustrophobic in Apple’s ecosystem. That feeling comes back whenever I stumble on a new product category or a place Apple seems to neglect, like smart homes. The walled garden works great when it works, but it can be maddening to watch Apple drag its feet on inevitable product changes or releases.

   The default apps on all Apple devices are perfect examples of this; Contacts, Mail, Calendar, and Reminders incrementally add features but only after years of neglect. While these apps lag, Apple continues to build tentpole features like contact posters around these apps, forcing you to ignore cool new features or use a lesser than product.

   “You’re no longer competing purely on the merit of the product,” says Carl Pei, co-founder of Chinese electronics maker OnePlus Technology Co. and the new smartphone startup Nothing Technology Ltd. If a person owns both an iPhone and an Apple Watch, Pei says, the chance of getting them to leave iOS is incredibly low. He adds that the Apple-only iMessage service has become “basic infrastructure” of communication and forces a limit on how much new mobile players can grow without it. A Google spokesperson said in a statement, “We believe it should be easy for users to switch between devices and platforms whenever they choose, and we find it frustrating that these principles are not equally shared by all platforms.”

   The results can be absolute bliss when Apple taps into the integration potential between its products. AirPods are one of my favorite recent Apple devices in years, and when it works, these headphones feel like absolute magic. FaceTime’s continuity camera magically taps into the iPhone’s cameras for the AppleTV and Mac to use or Universal Control which simply allows someone to move a mouse cursor from a Mac to an iPad. I always assumed that Google, with its profound control of Android OS, might build a similar molt, but they seem unable. As Apple makes these integrations deeper and deeper into the ecosystem, you have to wonder if anyone can compete because Carl Pei is correct. I’m no longer buying an iPhone because it’s a “phone” but for its deep integration across its ecosystem.

   Apple and Google invested resources in making the switch easier—the first app that Apple ever published in Google’s store was its 2015 “Move to iOS” app—but the paths remained bumpy. A designer who was working for Google around this time recalls that studies on customer feedback showed ex-iPhone users were frustrated by the process of moving media to Android and confused about leaving iMessage. The worse the first impression was with Google, the more likely people were to give up and switch back. This designer, who, like many insiders interviewed for this story, spoke on the condition of anonymity out of concern for career reprisals, remembers one regretful Android tester saying, “I want my life back.”

   Last Christmas, my friend moved from Android to iPhone using Apple’s migration app. I blogged about it here, but I was impressed at just how good of a job the app did in deep copying everything, including the system sounds. If anything, it did too much and changed standard iOS defaults to reflect settings from Android.

   What was more stark was how complicated iOS and Android have become and how many features feel as if they are universal between the two platforms. I’m curious to try the inverse process and migrate one of my iOS devices to Android. I may have to do that when I find some downtime.

3. Mindgrub and the Baltimore symphony Orchestra present AI in A Minor

   

   I’m beyond excited to announce AI in A Minor! The Mindgrub team and I have spent the last few months working to generate music and transform it into sheet music the amazing musicians at the BSO can perform. It feels incredible to know that soon you will have a chance to see what we’ve been working on.

   I also can’t ask for a better team than the Baltimore Symphony Orchestra, Greater Baltimore Committee, and Amazon Web Services (AWS) to help make this happen.

   Join us on August 9th!

   Oh, we’re still looking for sponsors and anyone interested in setting up a booth in the BSO hall. If you want to buy tickets get them here!

4. Meta Threads Countdown

   It appears 9to5Google got it’s hands onto an early APK release of Meta’s Twitter competitor Threads in “Threads, Meta’s Twitter clone, starts launch countdown, plus a few details on how it works“:

   Your Threads profile is also strongly connected to your Instagram profile. The two use the same username and display name, and it seems your Threads profile picture may have to be from Instagram. Additionally, anyone you block on one service is also blocked on the other.

   I shouldn’t be surprised by the tight coupling to Instagram, but I am. The coupling between Facebook and Instagram has always felt forced and as if they stifle the personalities of the different platforms. Threads (as I expect it) will be heavily text-focused, while Instagram leans into photos and video. How often will cross-posting happen? 

   Another unique aspect of Threads that many have been anticipating is the way it can connect to federated social networks like Mastodon (collectively known as the “fediverse”). It seems that Threads may not be ready to launch its fediverse features right away.

   Soon, you’ll be able to follow and interact with people on other fediverse platforms, like Mastodon. They can also find you with your full username @username@threads.net.

   The only other detail we could uncover about Threads’ integration with the fediverse is that if you choose to restrict replies on a post, it won’t be shared outside of the Threads app.

   When you limit replies, your thread will not be shared with your fediverse followers.

   Threads’ use of ActivityPub to connect into Mastodon and the collective Fediverse has long been a big question. In my newsletter, I compared Mastodon and the Fediverse to a network of towns, where each city has its form of government and content moderation rules. Threads’ appears to be a gated community that may allow its users to leave the gates and interact with others but still keep exclusive content limited to those within its gates.

   This social experiment will be interesting, especially when a metric ton of Meta users who first interact with the larger Fediverse through Threads and branded “@username@threads.net” name. I hope the other cities play nice.

   Digging deeper into the code, our team has also found that Threads may indeed have a web app. At the very least, we’ve found that the service’s profile links will look quite similar to Instagram profile links, simply appending your username after the base “threads.net/” URL.

   I assumed the animated website for threads hinted at more than just an app.

   

5. Screen Scraping

   Gizmodo has a piece on “Google Says It’ll Scrape Everything you Post Online for AI“:

   One of the less obvious complications of the post ChatGPT world is the question of where data-hungry chatbots sourced their information. Companies including Google and OpenAI scraped vast portions of the internet to fuel their robot habits. It’s not at all clear that this is legal, and the next few years will see the courts wrestle with copyright questions that would have seemed like science fiction a few years ago. In the meantime, the phenomenon already affects consumers in some unexpected ways.

   Twitter’s crazy rate-limiting meltdown and Reddit’s push to charge for API access are about one thing, AI data models. These systems are hungry for data, and access to that data will be vital to building the best AI models. Unsurprisingly, Google is making it known that as it ranks and offers prime search engine placement, all that delicious data is free game to them. When APIs become closed, people result to screen scrapping, and screen scrapping ends with paywalls and Twitter style rate-limiting… Wonder how this all plays out. 

6. Cord Cutting

   From Variety’s “Cord-Cutting Hits All-Time High in Q1, as U.S. Pay-TV Subscriptions Fall to Lowest Levels Since 1992“

   Cable TV operators’ rate of decline in Q1 reached -9.9% year over year, while satellite providers DirecTV and Dish Network fell -13.4%. In addition, so-called “virtual MVPDs” (multichannel video programming distributors) lost 264,000 customers in Q1, among the worst quarters to date for the segment.

   “The picture is not one that suggests that a plateau in the rate of decline is coming any time soon,” Moffett wrote.

   It is no surprise that I cut the cord years ago, but two things make me debate that decision every year, live sports and local TV news. I’m a transplant to Baltimore, so my team, the New Orleans Saints, is only available if I pay a ridiculous cost for the NFL’s Sunday package. As the team grew in national fame, I spent more but still could not watch big national games or occasional Baltimore Ravens vs. New Orleans Saints matches because of antiquated TV rights rules.

   In New Orleans’s I know local TV news like the back of my hand, but as a cord cutter, I can’t tell you a thing of Baltimore’s local stations. Why is this when it seems clear that traditional TV has lost the war? Why do holdouts like the local news and sports continue to make it hard to take the leap?

   In 2022, Forbes reported that “Streaming Viewership Surpasses Cable For First Time, Nielsen Says.”

   While streaming will likely remain the dominant form of television consumption, Fuhrer told Forbes that broadcast and cable will likely “see some rebound” this fall, as college sports and the NFL season start up and attract a higher share of viewership to those mediums.

   Outside of the NFL and college football, every other sport has embraced streaming and has worked to make watching games more accessible. American football is stuck with a problem, few companies are big enough to lay out the dough they require, and until then, football has dug in and embraced traditional TV.

   TV news is different but similar. News is stuck with an unclear business model in streaming, but something that, while declining, still pays. CNN pushed hard to grow its online news business only to scale back drastically. Other broadcasters skirt the concept but struggle with reducing the size of their audience or being limited to building their distribution systems. So while traditional TV holds on for dear life to maximize profits and return shareholder value, these companies continue to miss the entire point.

   In 2018 the Guardian reported that “Young people are not watching TV news, but they still want to know about the world,” and why wouldn’t we expect that when in 2012, the Holly Wood Reporter spoke of these issues in “Why Kids’ TV is Scrambling to Stay Afloat“.

   Despite the effort, Nickelodeon and others increasingly compete with their own content on Netflix as their parent companies eagerly make rich licensing deals. In early May, Bernstein analyst Todd Juenger issued a report blaming “drastic declines” in ratings for kids networks in part on repeats of older episodes in homes that subscribe to the streaming and DVD service. The study was controversial because it was based only on data from homes that subscribe to TiVo, which is not necessarily typical of the broad universe of TV homes, but Juenger says “executives should think hard whether they want to sell this content to Netflix. The money looks good in the short term, but if you believe what the data says, as Netflix gets more subscribers and people who use it more get accustomed to it, the impact is going to grow.”

   Local TV, cable news TV, and football are getting the profits today, but both are missing out on establishing a relationship with a digital-first generation they’ve chosen to ignore or force to meet them on their turf. As Netflix begins its cord-cutting tour, do not forget Netflix CEO Reed Hastings’ quote on who they compete with:

   “We earn consumer screen time, both mobile and television, away from a very broad set of competitors,” the quarterly earnings statement read. “We compete with (and lose to) ‘Fortnite’ more than HBO.”

   What will the next generation pick when deciding between Roblox, Fortnite, and the NFL? Guess we will find out.

7. Reddit and end of Open API’s

   

   
   This sucks. Apollo has been my go-to Reddit reader, and I don’t want that to change, but come the end of the month, it’s happening if I like it or not.

   If you’re not in the loop on Reddit’s API drama, the TL;DR is Reddit moved from open and free APIs to a fee-based system that charges based on the number of API calls you make. If this sounds familiar, it’s because Twitter went down a similar path, and many other open platforms have decided to shut the doors to open API access. The argument for why? OpenAI and AI models are being trained on hordes of open Internet data and, of course, the possibility to eek some revenue out of all the folks hooked on Reddit’s content.

   As you might imagine, that approach makes the cost to run something like Apollo unsustainable.

   Is Reddit wrong? Apps built for platforms like Twitter and Reddit are like symbiotic bacteria, but one organism is much more dependent on the other. As a platform, Reddit is about user-generated content, and as with Twitter or LinkedIn, it makes us feel like investors or partners in this whole social sharing experiment. But let’s be honest. Revenue and control of the platform you own is what this is really about. If you’re not in control of the last mile, you can’t control how your consumers interact with you or it. You’re constantly limited in how you can advertise, how you personalize, and the ways you can use them to generate revenue.

   Hey Reddit, when you fix the mobile and iPad apps, call Mindgrub. We make great mobile apps.

8. Open API’s

   The idea of open APIs and access to platforms has become a surprisingly divisive thing. Like most stories in 2023, our story of APIs starts with Elon Musk and Twitter and the decision to shut down third-party app access. 

   Many, many, many folks were upset that Twitter would shut off access to TweetBot or Twitterific. These apps have been part of Twitter from the start and one inspired Twitter’s logo. To add insult to injury, this made us all collectively realize that Twitter’s mobile app is not great (call me, Mindgrub builds excellent apps). But, Twitter didn’t just band third-party apps – what it did instead is rate limit API calls and implement a new system to charge based on the amount of API calls per month. The price tag was so hefty that bit by bit folks said nope.

   Unrelated to Twitter, OpenAI blew through the doors of technology like the Kool-Aid man. Whoever had AI on the 2023 Bingo card deserves all the money. The products like DALI-2 and ChatGPT continue to blow all of our socks off – but then the deep dark secrets of OpenAI and other AI platforms began to drip out.

   These LLM (Large Language Model) systems need data, and when I say data, they need all the data. The more you can feed the dang thing, the better – it’s like Seymore, and it wants all the information. Some of this best information came from the most open of sources, places like Twitter, Reddit, and Stack Overflow. These platforms are unique in havings tons of experts who share their advice or answer questions in the most open forums.

   Elon Musk and Twitter responded that this was why they needed to lock down APIs and tweets so that eager AI training models won’t try to consume this valuable training data without paying the troll toll. Reddit and other sources of these models followed, and now we find ourselves full circle.

   Apollo, my preferred Reddit reader and the only reader with a pixel pet finds itself facing the same issue as Tweetbot and Twitteriffic, the costs for the APIs they need to use from Reddit cost too much.

   I get it. I understand it. But sometimes I think of the founding of our great Internet and a time when information was free and people linked to link for love. I guess that was the Internet’s 70’s – and today is a differnet time, but I can’t help but wonder if the hordes of people training models on the open Internet might find the reigns getting pulled a little tighter. I also wonder if this just continues the trend of pay walls popping up everywhere.

9. Originality

   AI has moved the idea of derivative work into the headlines. After all, originality makes the greats great, or does it?

   “Great Artist Steal” – Steve Jobs

   I mean, think of all the original work we love—music, movies, tv, and art that genuinely shaped us.

   

   Art is about the creation of something, and something sometimes comes from something. That something original becomes in itself a new art form.

   The best part is sometimes a cover, a recreation of what we love and believe – but it also comes from sampling what we like and love but just the right taste.

   But don’t go too far. A sample is good, but too much could be a copy.

   But let’s not forget. Real artists steal, right? And art is about creating something not just new but from what we learn or acquire. I learned to write from reading and typing – Now is the time for all good men to come to the aid of your country. 

   I mean, when do we, as humans, get caught up with the past? Not us. Nostalgia never gets the best of us – but even if it did, we would make it better, and better, and better, and better, and better, and better, and better, and better, right?

   So how dare AI copy all these original ideas we have!

   I think we need to protect everything and keep the web or anyone from knowing things like a world-famous authentic New Orleans Red Beans recipe. 

   So is generative AI any different than our own derivative work? I mean, we can’t seem to figure that out – and sometimes, even some folks can’t find the fun in a little joke. Ideas come from so many people I know, so much I’ve read, and so much I want to know.

   But maybe DALL-E and Open-AI find it kind of funny?

   

   Originality is a really complex topic, and the impact of AI on jobs is scary. Shouldn’t we want the same inputs that allow us to create to be the things we feed the AI we use? Is that original work generated from the same tweets and arrives I read as derivative as my own?

   In the end, we can all agree that copying can create some of the best art – or maybe we’re all just here for the ride.

10. Security Best Practices When Using AI to Write Code

    At Mindgrub, the engineering team, like many, has found itself wondering just how good AI is at writing code. What do these things really know, and can they make us better, strong, and faster?

    I’m here to tell you that it is pretty good, in some cases damn good. If you need a quick code snippet or find yourself wanting to convert existing code into a different language, tools like ChatGPT do a fantastic job. We’re also actively using and exploring tools like Github’s CoPilot – a development assistant that makes intelligence look antiquated. Our engineers are also investigating a metric ton of generative AI code tools like CodeWP for WordPress, AWS’s CodeWhisperer for, and X.

    Generative code is impressive and quick, but is this code safe to run? After all, these AI tools open a brand new and wholly unexplored set of security concerns and unknown vectors for attacks by hackers. Most of these may not directly impact code security, but they point to the level of awareness we as technologists need to have as we explore the use of these platforms in our work lives.

    Developers have already built AI tools that can brute force or perform login stuffing with an accuracy and speed that is impressive. All of this uses open-source tools like PassGAN, tools that are getting better every day. Some researchers have gotten clever and jailbroken or avoided AI safeguards to trick systems into writing code using known exploits or to write code that is used for nefarious reasons such as a DoS (Denial of Service attack). Others are creating advanced phishing systems that create highly personalized messages making truth and reality increasingly hard to differentiate.

    Can we trust these types of tools to our most junior engineers or non-engineers to create code for production? Do we have a choice? We all know the reality is this is already happening and only going to increase. What we really need is to find ways to keep our AI-assisted code safe and secure.

    So to help, I will first explore using a handful of these tools and give some hints along the way. I will also offer suggestions on what to look for and ways to keep the most generative code secure.

    Github CoPilot

    Many in my age range have joked about CoPilot being the replacement for Clippy, but both are assistants. However, CoPilot, unlike Clippy, is an assistant powered by OpenAI’s GPT artificial intelligence and trained on bajillions of code hosted by Microsoft’s GitHub. These days Github is the apparent elephant for public and private code repositories. It is also home to an incredible amount of open-source projects. If schools are code repositories, GitHub is Xavier’s school for the gifted or Hogwarts, without the riff-raff.

    CoPilot integrates into many popular IDEs, such as IntelliJ and Visual Studio, to extend intelligence or auto-suggestion feedback. For most, it will feel like you’re getting a quick suggestion based on context – but these aren’t old-school suggestions. Often, you will find that CoPilot will have suggestions that are entire functions vs. finishing a line or two of code.

    In this, CoPilot and tools like ChatGPT can be very different. CoPilot feels more like an assistant or peer programmer offering thoughts along the way. You pick and choose, but the architecture and direction of development are still very much you.

    As a code generator, the results are mixed. In 2021 a DevSec engineer reviewing early results provided multiple examples of code that were prone to suggest code with several security issues. My experience is mixed. I’ve witnessed code snippets with SQL injection vulnerabilities or other minor problems. The more significant concern IMHO was not the quality of the code but the speed at which I accepted that the code would do what I anticipated.

    OpenAI ChatGPT

    OpenAI’s ChatGPT is what we now refer to as AGI or Artificial General Intelligence. For example, GitHub’s CoPilot has been primarily trained or concrete knowledge bases around programming and code giving it intelligence that is limited to a very particular realm. In short, it’s like a toddler who can tell you everything about Pokemon and nothing about the general makings of our world.

    AGI makes writing code more of a hobby for ChatGPT but also gives it the ability to be a bit more creative in how it answers questions. It can add data from its general knowledge like we do to come to sometimes surprising conclusions.

    ChatGPT, as a development tool, is an excellent starter. It excels at transforming example code into your preferred programming language. It can take code snippets and re-write them with additional features or adjustments. It also does a fantastic job of creating starter applications.

    To test CoPilot, I asked it to generate the full login, logout, and sign-up logic for an application in JavaScript. I purposely omitted details to see if the AI would assume the need for a unique identifier like a username or email address. I also avoided mentioning the need to sanitize or encrypt arguments and SQL values like passwords.

    Weirdly the code response differed. Each prompt could generate a wildly different answer. My first attempt at my prompt displayed an application that hard-coded the database password and showed a noticeable lack of validation. That request failed to complete as if the AI hit a point that it knew invalidated the previous response. By not updating my prompt and allowing it to regenerate, I got a much better response that essentially fixed issues without me editing or asking.

    Like Github’s CoPilot, the results came back mixed, but most of my code from ChatGPT required a bit more knowledge and cleanup to run. For example, ChatGPT suggested a make file and a SQL script for my user database table but did not help me actually do the task. It was much more of an accelerator, requiring me to reimplement a lot of what it provided.

    How do we keep it secure?

    AI generative code tools are super accelerators. These tools are also trained on our own lousy code and suffer from the human mistakes we are all prone to. For junior engineers and non-engineers alike, these tools provide incredible power, but they will not mean the end result is better, more secure, or better quality.

    So how do we keep them secure? We do what we should have (or what we are already doing).

    First, let’s keep following the best practices of software development. If you have a team, make sure you enforce peer reviews and merge reviews. As the saying goes, measure twice and cut once – more eyes and especially those of a lead or senior engineer, will only make your code better.

    Second, good unit tests and code coverage are some of the best checks a developer can put in place. Unit tests require the engineer to understand the expected results of the code they write and to verify that the code reacts as anticipated. By requiring larger code coverage, our engineers can use more generative code, but we can safeguard the upper and lower limits of these operations with these tests. 

    Liscense and dependency management can accidentally pop into code when using ChatGPT it’s not uncommon for it to recommend libraries and incorporate those libraries into a larger code base. For production code, this can unexpectedly force code to accept a GPL license, open, sourcing a chunk, or introduce vulnerabilities in an older library. These days we can add analyzers to our CI/CD pipeline that check and warn for these scenarios and reduce unexpected risks.

    Other tools in the CI/CD pipeline also open the door can also safeguard against bad code quality:

    •  Lints and code syntax checks help maintain code conformity and check for common mistakes in a language. These same tools can scan for passwords checked inline to a code repository and reject code not in the company’s agreed-upon format. 

    •  Many companies offer code security analyzers that look for the common mistakes and prevent developers from 

    •  Static code testers scan the executable binary generated from an application for 

    If you still find yourself adament that your company or project is not ready for generative AI, you can also look into several tools that help detect AI-generated code. As a warning, this can be a bit of an arms race. As new AI tools improve, the detectors will take time to adapt and identify the latest version of GPT or CoPilot.

    For many of our dev shops AI will introduce a new wild card in how we build things – but that wild card can be a great accelerator that increases productivity and helps make junior engineers bigger contributors to production projects. Embracing the unknown can be scary, but with the proper safeguards in place, we can create a secure environment where our teams can thrive.