Santa Cruz Works and Santa Cruz County Office of Education are proud to announce the first place winner for the Best High School Project at Cruz Hacks 2022: Remedy. This all girl team enjoys learning at Stanford Online High School / Homeschool, Davidson Academy Online, and Davidson Academy Online, respectively.

Inspiration

Skin cancer is the most commonly diagnosed and preventable cancer, for some types of skin cancer, discovering its presence early on can increase one’s 5-year survival rate by over 10%. However, it still causes tens of thousands of deaths in the US alone; how is this possible? Especially during times of Covid, many patients find it impossible to schedule a checkup with their physician. REMEDY is a checkup in your pocket. It’s a checkup everyday. It’s a checkup to reduce the workload of our already over-worked healthcare workers. It’s a checkup that is convenient to do everyday. It’s a checkup that anyone can do from anywhere in the world with little more than a mobile device.

What it does

REMEDY is an AI trained with over 27,000 photos of various skin conditions, from eczema to melanoma cancer. Our original idea was to track the progression of various skin diseases and automatically return the improvements or worsening of the disease so that users can gauge their progress. However, with a lack of data on the progression of skin diseases, definitely not enough to create an accurate AI, we have settled on creating a slightly simpler program. It analyzes a picture the user submits in the app of a possible disease site, and will be returned with the expected condition and percentage that the AI’s analysis was correct. The page also allows the user to create a link to their results, so they are able to send it to their dermatologist or family members. These programs work in accordance with one another to allow the user to get the best on-demand checkup, right from their pocket. There are additional pages in REMEDY mainly to create the best user experience. For instance, the settings page allows users to change their viewing preferences, whether the app is in light mode or dark mode.

How we built it

We used a number of technologies to create the finished product. From the design standpoint, we used Figma to plot out the app wireframe and Adobe Illustrator to create the logo. We worked on the code collaboratively using VSC live share, meanwhile Github served as our form of version control. To create and train the neural network, we used Apple's CreateML platform. Additionally, we turned the AI model into an API using Socket.IO, and Express. On the topic of Express, we also used it to host the REMEDY stat viewer. We used Cordova for mobile development, cutting down the development time of the mobile app significantly.

Challenges we ran into

Our dataset contained 27,000 files in order to produce an extremely accurate program to detect 10 varieties of skin diseases. When we started developing the AI, we realized that it would’ve taken days to train the data on our relatively old laptops. We were definitely worried that we wouldn’t be able to finish the meat of our project as a result of the lengthiness of training ML. Luckily, one of the members had an M1 Apple computer, which means we were able to train the model many times faster on that computer with its 7 GPU Cores. We reduced the time from months to just an hour or so with this shift. We still had to sacrifice the amount of epochs so that we could cram in the amount of training time within the time frame. 

Accomplishments that we’re proud of

We’ve used various technologies for the first time in this hackathon, from Cordova to Core ML. Our success in creating the product we envisioned with all these new technologies is significant for us. View the section below for more information on the new frameworks that we learned and incorporated into REMEDY. 

What we learned

We learned a great deal throughout this hackathon. There were many firsts for us in developing this app, and we feel as though we have really expanded our knowledge. It was our first time designing and coding a mobile app, our first time making a light/dark mode, and our first time using Cordova, Core ML, and Keras. Our team is very proud of what we’ve accomplished through the construction of REMEDY and how we’ve all grown our skillset.

What's next for REMEDY

In the future, we plan to add more skin diseases onto the catalogue so that the app is even more broad. We want to give it more time to train with more epochs. As part of this process, we also want to use patient scans to make the AI smarter by adding their photos to the database of diagnosed skin diseases. This could be achieved by having a special staff of medical workers verify if the AI’s results were correct, or if not what they actually should have diagnosed. We also want to make REMEDY more accessible to people around the world by translating the app into many different languages. As mentioned before, our biggest hope is to be able to track the progression of many skin diseases. The future may include working with hospitals to collect said data in the hopes of helping many.