I'm halfway through my summer internship as a Creative Technologist at Deeplocal! Deeplocal is a creative technology and experience design agency that brings brands to life through interactive experiences. Their studio in Pittsburgh, PA has a 12,000 sqft prototyping facility, so they're able to see projects through from a concept to a polished final product. During the past 6 weeks I've mostly been working on the production phase of one project, although I've had a few opportunities to dabble in concept, research, and pitching to clients.
[One of my favorite past Deeplocal projects: Never-Ending KIT KAT® Trick-or-Treat Bowl, as featured on The Late Late Show with James Corden]
It feels crazy that after just one year at ITP I've picked up enough skills to meaningfully contribute to a physical computing-heavy project. I've drawn on my software engineering and web dev background for sure, but I've also completed plenty of work involving microcontrollers, sensors, and general infrastructure setup that I wouldn't have been able to make sense of a year ago. There's always more to learn and I've continued to learn on the job at Deeplocal. Here's some tools and concepts I've gotten familiar with in the past few weeks, and although they may be a bit dry they've helped me level up as a Creative Technologist.
[Inside the Deeplocal office]
Raspberry Pi
Raspberry Pi is a very cheap, very small computer, perfect for embedding in physical computing projects. They're huge in the physical computing world and I don't know how I haven't used one before. Although to my parents' credit, they definitely tried to get me into Raspberry Pi kits as a kid. At Deeplocal, we're using Raspberry Pis inside of a product to run a screen-based UI and local server. I wrote a full-stack web app with React and Express that responds to some serial data from another microcontroller, and that app is displayed on a small monitor connected to the Pi via HDMI. So using web tools I'm already familiar with, but running them on a handy new platform.
[Raspberry Pi board]
Ansible
It's been part of my job to configure several Raspberry Pis for our very specific dev environment. There's a lot of setup involved here to install tools we need, create a filesystem, grant permissions, etc. So many steps that doing all of them in the right order for several boards would be a nightmare. Ansible is a convenient way to automate this configuration process, in the form of a playbook that we run on new boards to take care of all of this tedious setup. The coworker who wrote this playbook seems to be a Linux + Ansible god so I've been relying on his scripts and have only dipped my toes in this topic, but I want to learn more about what it can and can't do. For example, there's a few setup steps that we haven't automated, like cloning a git repo onto the Pi and adding .env files.
pm2
Another part of the Raspberry Pi setup was getting our code to run automatically on boot. Every time the Pi powers up it should just run our server and UI with no fuss. pm2 is a process manager for Node.js applications (such as a React app or Express server). It lets you run processes in the background and keep them alive forever. Helpful when you want to run a server AND a web app AND a browser window to view that web app AND still have the commandline free for debugging. Part of the Ansible playbook uploads a pm2 configuration to the Pi that tells it to launch certain processes on device startup. I've gotten a lot of practice with starting and stopping processes, viewing logs, and configuring start scripts... Like, a lot of practice.
[An example of pm2 processes listed on the commandline]
ngrok
ngrok is a tool that lets you expose your locally hosted web server to the internet. When I'm developing a server I only run it on my laptop so I can quickly test and make changes. Other apps on my laptop can make requests to that server via its local address, like http://localhost:3000. But it can be helpful to let other apps on the internet also use that address for testing, before deploying the server somewhere. ngrok makes this possible by creating a temporary web address that tunnels to your local address. I used this for testing some endpoints I used with Twilio and Supabase (more below!)
Webhooks
This is more of a concept than a tool, but it's something that's been pretty abstract to me until now. The standard definition of a webhook is "a user-defined HTTP-based callback." That didn't mean much to me until I needed to send data from a server to a client. When working with an API it's usually the other way around, where a client polls a server for data-- meaning the client spams the server with requests for data until it gets something. With webhooks, the client can define an HTTP endpoint that the server can hit with a POST request to give it some data. This prevents the need for the client to poll the server, the server can send data when it wants. I used this to send data from an Express server to a React app, and to let a microcontroller trigger some pre-defined web actions (again, more below!)
[Diagram of how webhooks work]
Twilio
Twilio is a popular product for programmatic communication. You can purchase a phone number and use their APIs to make phone calls, send and reply to texts, and more. A lot of text campaigns or appointment reminders use Twilio, texts where you might have the option to text "STOP" to unsubscribe or "YES" to confirm. Their platform is incredibly easy to use, I was able to use their node.js library to text myself in just a few minutes. I wrote a small Express server with an endpoint to let our Twilio phone number reply to incoming texts, and ngrok came in handy for testing this server before deploying. The Twilio web console provides an option to set up a webhook (!!!) to let the phone number respond to texts, so I gave it the URL generated by ngrok and appended my API POST endpoint path.
Google Cloud Platform - Serverless Functions
But! ngrok is only temporary for testing, the server still needs to be deployed somewhere so it has a permanent web address. In the past I've used Heroku for hosting my small servers, but Deeplocal likes to use the Google Cloud Platform ecosystem. I got some practice with GCP serverless functions, which are perfect for hosting webhook HTTP endpoints. I wrote a few serverless functions to let a microcontroller hit Twilio and database functionality. This felt similar to Heroku (where I host this blog's server) but with the benefits of other GCP tools (ex: storage buckets) and only paying based on usage.
Supabase
Supabase is a backend-as-a-service platform that lets you get up and running with a PostgreSQL database within minutes. It's an alternative to Firebase that is perfect for storing structured table-like data to query with SQL. I don't have much else to say here except that it feels easier to use than heavier platforms like MongoDB or Firebase, and the UI is slick.
[A Supabase data table]
Particle Boards
The only microcontrollers I used at ITP were various types of Arduinos, which is a brand. I had no experience with other microcontrollers until coming to Deeplocal, where I've had the chance to use Particle boards. Particle is a platform-as-a-service, providing hardware, software, and cloud services to help create IoT (Internet of Things) projects. Basically made to power any tech project that needs to connect to the internet or have a cell connection. The Particle web console is amazing and I've been using it to manage a fleet of several boards for a project, for example
- OTA (over the air) firmware updates for the fleet
- Remotely triggering a function on a board
- Integrating with webhooks (!!) in just one line of code
- Checking cell data usage stats for several boards
I'll definitely be buying one of these boards when I go back to ITP.
[Example screens from the Particle web console]
RFID
I had one afternoon to experiment with RFID, which seemed like magic to me before diving into how they work. One example of RFID readers and tags is arcade games where you tap an arcade card to play. Maybe I should have understood some of the magic from the acronym RFID: Radio Frequency Identification. Here's how it works, as I understand it:
- An RFID tag, like the arcade card, has a microchip that contains some identifying data
- An RFID reader has an antenna and emits an electromagnetic field
- When an RFID tag moves in the range of this field it receives enough power to activate itself (that's magic)
- When activated, the RFID tag transmits its identifying data to the RFID reader through radio waves
- The RFID reader decodes this transmission and sends the data to a local server to handle
- I can code that local server to do whatever, like use the RFID tag ID as a database key
Lots of things to experiment with here, but my immediate thought was how to embed RFID tags in Jello.
[An arcade RFID card reader]