Project Shark
As a tradition, almost all Grasshopper3D and Rhinoceros3D plugins are given animal names that somehow relate to what the plugin’s function is. I’m not very sure about how this started, but I think it just brings a little fun into the environment. Following this tradition, names like 'fish' and 'trout' came to mind when I was working on a plugin that allowed real-time 'streaming' of geometry to other platforms (stream; water; fish). Luckily, a friend advised that I should consider something cooler, like 'Shark’. Shark3D was already the name of a computer graphics software, so I settled for Project Shark, since it was one of my very first major software projects.
During my time working at rat[LAB] Studio, I spent a lot of time looking at XR technologies in the design industry, primarily Augmented Reality. After a lot of experimenting and research, I arrived at Unity3D with the Vuforia package as the most accessible and reliable way to create AR applications (at the time). During a conversation, Sushant Verma asked me something along the lines of “Can’t we import the geometry from Rhino into the AR app directly?”.
While I didn’t have a solution for that immediately, it sparked a lot of curiosity. I spent my free time in the subsequent months learning C# programming for Unity3D, Network communication, Mesh geometry and Data compression. Surprisingly, I didn’t find many solutions that let you stream geometry data over a network directly, or even compressed geometry data efficiently. This meant that I had to write my own Frankenstein method to achieve realtime compression and streaming of geometry data. Studying open source software turned out to be a great way to learn complex programming ideas. I found a great proof of concept by Junichiro Horikawa on YouTube that showed mesh geometry from Grasshopper streaming in real-time to Unity3D via UDP communication. There was a lot that would have to be added on top of this, of course, but this gave me enough encouragement to continue developing the project.
In some time I discovered Fologram, which was essentially doing the exact same thing I was planning to do. As far as I could tell, the Fologram client app seemed to be using Unity3D as well (not sure about this). So if I was to complete my project, it would basically be competing with this awesome company that’s doing great work already. I lost a lot of motivation at this point and even considered simply applying for a position at Fologram.
I had worked on the project to a point where it was being tested by design and tech professionals from all over the world, through an invite-only beta testing program. But after that I seemed to have no time to spend on the project (or at least not enough will power to make time for it). With some motivation from my close friends and a new job that gave me enough free time to spend on completing Project Shark, I managed to re-write the entire system from scratch. This time I had more experience and knowledge along with useful feedback from the beta testers.
For mesh data serialisation and compression, I chose to convert all the data — vertices, faces and UVs — into a proprietary format before serialising it using MessagePack. Some data was simply stored as a serialised JSON string, to ensure the possibility of feature addition in future without worrying about backwards compatibility. In terms of performance, I achieved a reasonably good compression ratio (40–60%). I may work on another method for mesh serialisation in future that would improve on the current version. I also considered using various types of geometry types like boxes, spheres and other Rhino data types to further reduce the amount of data that had to be serialised, but that meant a whole lot of more work on both ends and more chances of bugs for a seemingly small performance boost at this stage.
The server for Project Shark was written in JavaScript with Node.js and hosted on Heroku, since I didn’t have a dedicated hosting plan. This could be seen as a security flaw, but I made sure that no personal data was being transmitted over the server and all the geometry data was end-to-end encrypted. The Grasshopper instance requires the unique identifiers of the client applications in order to stream the data to them. Also, the method of serialisation is entirely proprietary, so unless someone knows exactly how to deserialise the data and extract the geometry from it, your geometry is safe. Hosting on Heroku meant that the performance will be less than ideal, so I offered to set up a dedicated server and custom client apps for corporate clients. The server basically acts as a simple relay between the Grasshopper plugin and client applications, passing the data to the correct client from the correct source.
Deserialisation and another conversion process had to take place in Unity3D for the client app. Once MessagePack deserialised the incoming data, I procedurally reconstructed the mesh geometry in Unity3D. This process can be further optimised with a diff calculation at some point and other methods. I faced some trouble in getting MessagePack to work on Android, but eventually figured it out. Once it was ready, I made the Android app available on the Google Play Store for a very small price so that people looking for random Shark related apps and games don’t download the app and give it a bad review. iOS users have to wait a little longer for Project Shark unfortunately (I hate Apple).
I added the ability to change parameters from the client app, enabling 2-way communication and made many other major improvements. Project Shark is still not 'complete' by any means, but it is now available to download and use as a complete solution to real-time AR for designers. I would like to add compatibility with more modelling applications like SketchUp, Revit and Blender, followed by a wider range of client apps — VR, Web and even MR. The aim is to extend the modelling viewport in XR and make these technologies more accessible to designers, without a steep learning curve.
The budget for Project Shark was basically zero (except for a Vuforia license I had purchased and ended up using for other projects) and I was the only one working on it (except for some of my friends and juniors who helped with graphics and stuff). It helped me acquire more clients and projects, enabling me to leave my job and start ARPM Design and Research.
Project Shark is available for download on Food4Rhino and Google Play Store. It will continue as a long term development project at ARPM Design and Research. Feel free to reach out to us if you would like to contribute to it!
