Artificial Intelligence coupled with Apple’s latest tech allows you to create captivating artwork using pictures from your iPhone or iPad.
I downloaded Looq on a Saturday morning a couple weeks ago. That day, I’d written a long list of chores I needed to take care of—cleaning a stack of dishes, laundry, a trip to the store.
But once I started playing around with the photo editing tools in Alexis Creuzot’s iOS app Looq, that to-do list went right out the window.
Here’s what makes Looq so fun. The app leverages the power of on-device artistic style transfer to turn your photos into instant masterpieces, in the styles of famous artists. Looq comes with 10 styles built-in, with the option to upgrade for more (and all future) styles.
I have a lot of pictures of my animals because they’re always just being so cute, and I can’t help myself. Using the intuitive and seamless interface, I was able to add the amount of the style I wanted and have the style cover as much (or as little) of the photo as I wanted. Here’s a quick screen grab that shows how easy this is:
My personal favorite style right now is “Crayon”, but it’s really fun and easy to try out all the different styles on the same photograph.
Considering how much fun I’ve been having with Looq, I wanted to chat with Alexis about his inspiration for the app, what challenges he faced in developing it, and what’s on the horizon for Looq. Here’s my interview with Alexis, along with some sample photos made in the app.
What’s your background, and how did you end up building Looq?
After earning a Master’s degree in Software Engineering, I went to work in mobile development where I chose to specialize on Apple technologies. Working as an independent iOS expert, part of my job is to stay on the cutting edge of Apple software.
In 2017, Apple introduced Core ML at its annual WWDC event. My understanding of machine learning was very limited at that point, but I was quickly hooked.
I often try out new Apple frameworks by implementing small side projects. This is how Looq began, as a small test app for Core ML. As I kept returning to it, the project matured and eventually evolved into a full-fledged app. At that point Aurélien — my Android counterpart — hopped on the bandwagon to develop the Android version.
We released Looq in May 2018, and we’ve continued to improve it since then.
What’s does your tech stack look like, and what tools did you find helpful?
There are two main pillars of Looq’s tech stack.
To train the models, I have a machine running on Ubuntu 16.4 with Anaconda. The actual training is done on PyTorch, which I found to be the fastest and easiest to tweak.
Additionally, I installed ONNX and coremltools in order to export the PyTorch model to Core ML format. The iOS client is developed on a MacBook Pro with the latest MacOS and Xcode versions.
What was the most challenging part of developing Looq?
A first hurdle was the environment setup. It took me a long time to get all dependencies necessary to be compatible with each other. Anaconda solved most of those issues. Getting the machine learning backend to use the GPU was another tedious chore, which I solved after finding the proper driver and CUDA version for my graphics card.
Once the environment was workable, I spent some time trying different ML backend. Keras, Torch, and finally PyTorch. That last one got me the best results-over-time ratio, but it ended up also being the most complicated for converting its models to Core ML. The ONNX open format rescued me on this front.
From then on, it was mostly trial and error, tweaking the training algorithm and the graph layers to get the best results, which is of course quite subjective with NST.
The client app was less time consuming. Core ML does the heavy lifting, and the biggest challenges were optimizing the CPU and memory consumption. Since the app can handle fairly large images (which it can render for up to 20 styles), a lot of this needs to be cached on disk.
One last challenge was to reduce the size of the app despite the number of models it shipped. I was able to do this using Core ML 2’s quantization tool. Check out my latest blog post for more info on that:
Do you have any advice for other developers who are looking to get started with machine learning?
Machine learning by itself is an extremely wide field. As developers, there are more and more libraries and tools available every day to look into, thanks to AI researchers around the world.
My advice to a beginner would be to pick an existing ML model and try to create their own. There are plenty of open source projects on GitHub anyone can clone and fiddle with.
Once you start training your first model, you gradually pick up the elements shaping its predictions. You actually end up learning a lot more trying to improve the accuracy of your model than creating it in the first place.
What does the future hold for Looq?
We are working on new filters on top of enhancing the ones already available. Another functionality we’re considering is short video / live picture stylization. Overall, our goal is to continue improving the app while keeping it clean and simple.
We’re also looking into ways to get Looq on the map. Our main competitor, Prisma, can only apply some styles when connected to the Internet, and they launched their pro version as a subscription model.
We think that a freemium model with offline capabilities like Looq can be a strong alternative on the market.
Comments 0 Responses