Props Semi-functional Reach TacPad - Mobile Project

[Fallen]

I've been wanting to do a Carter build for several years now, but as part of that I'm working to get back in shape, so the bulk of that build hasn't started yet. Carter has that TacPad on his forearm, and it's always been in my mind that I wanted more than just a static image in place there. Since I have the time before I start on the armor parts of my build, I'm using this opportunity to start the process of making the TacPad something more. As some of you may know, I'm a mobile developer who does primarily iOS but also has experience developing for Android. I want to take that experience and develop a mobile app that I and others can use as a TacPad in the style of Reach with functionality.

I will warn you, that there will likely be some technical jargon in my posts. I'm a nerd, and I love all of the problem solving that comes with writing code and creating functional apps.

The Primary Goal
Build mobile apps for iOS and Android in the style of Reach that can be used to provide useful information throughout conventions.

Ultimately I want to make something that will track and display my health information throughout the day, give me reminders, and ideally be able to control various planned systems in my suit when I build it. I also want to make the app something that I can share with other members to set up and use for themselves with minimal effort.

MVP
First up I need to define what is MVP (Minimum Viable Product) for this project. My stretch goal has a lot of problems to solve and systems that I have never written code for, so I need to split things up into features and define a prioritized order to start getting things done. For this I've defined MVP as having the main UI based on references provided by TurboCharizard but with some kind of function rather than a static image. I've decided at this point to take what looks like 4 buttons on the left side and sync them up as a mini soundboard, and add some hard-coded information in what looks like a smaller info view within the greater interface.

The Technology
iOS: Swift, supporting iOS 13+ 15.0+
Android: Kotlin, support Android 8+ (Oreo+) DEVELOPMENT ON HOLD

Hoping to be able to use some Kotlin Multiplatform to share some of my business logic across both iOS and Android.

Testing
TestFlight Invite
I'll be doing private testing that I'll open up to any members who want to be involved sometime after MVP. Feel free to PM me if interested.

Shout Outs
TurboCharizard for providing images to reference creating the base interface
RandomRanger for additional functionality ideas
Angus314 for testing early features and providing feedback and suggestions

 

[Fallen]

Progress Tracker

Links
Update 1 - The View
Update 2 - Heart Rate
Update 3 - Step Count
Update 4 - First Pre-Release Builds Submitted
Update 5 - Definable View (attempt 1)
Update 6 - Nearing MVP

 

[Fallen]

Update 1
MVP is nearing completion. As of now both projects for iOS and Android are created with the basic TacPad layout. The blue "display" is holding some hard-coded placeholder values, and the four buttons are linked to 4 sound clips of Carter that I added to give them some functionality. Eventually I want those to switch value for my purpose, but just an idea of something that could be done for someone who maybe doesn't have lights and fans installed that they want to control from the TacPad.

Top: Pixel 3 on Android 10
Bottom: iPhone XS on iOS 13​

Recorded on iPhone XS. I didn't bother setting it up to show clicks, so I tried to hold on buttons a little longer. iOS by default darkens the text slightly on press.

And for reference screenshot of the forearm from the game.

 

[RandomRanger]

\0/ It's got a thread! Progress is going quick, loved the pics in discord. Once I reach that stage in my suit I'll probably be taking a look at the code

 

[Rock Lobbster]

I love this idea! You're doing one heck of a job on it!

 

[Fallen]

Not a big update to give, but I noticed a couple of issues with the layout on especially smaller devices on both iOS and Android. Last week I spent rewriting the iOS layout and this week the Android layout. Android is much harder to make exactly right because there are so many devices and screen sizes with all the different manufacturers. As a developer, the limited options for iOS devices is actually kind of nice for testing.

Next up is to start displaying some amount of health data. I've never worked with HealthKit or Google Fit before, so I'm looking forward to learning about what data they provide that could be useful and how they work on a technical level.

What I want to know is what are the types of health data you most want to be able to see if you were using this tacpad? I'm thinking of what will be displayed in the little mini-display (where the 61bpm is on update 1 images).

 

[PlanetAlexander]

Thinking of what it's like in heavy, hot armour, what about body temp and hydration levels? Don't know how possible it is with that technology.

Oh, and some cruel, punishing reminders to eat/drink would be SUPER helpful

 

[Fallen]

Added a poll to get insight into whether or not people are willing to help beta test the app.

iOS running iOS13+
Android running Android 8.0+

I just need to know a rough estimate of how many are willing to test. I'll likely post a link to join the beta group when the time comes.

 

[CrimsonViper97]

Mark me down for Android!

 

[Fallen]

Update 2 - Heart Rate Monitoring Begins

I haven't had a lot of time this past month to work on this project. Got onto a short project at work with an extremely aggressive timeline. We're talking building an MVP version of an app to be deployed in ~1month. Not a lot of time to go from idea to design to completion especially given there were several unknowns when starting. That should be winding down soon, and I got the chance to get a little bit of side project work time.

I'm diving into the HealthKit stuff on iOS in order to grab heart rate data and afterwards steps. In what I played with yesterday added a settings screen to be able to enable/disable showing heart rate as well as authentication to track heart rate information. I'm also able to start querying the heart rate data, but am focusing on fetching on some set time interval rather than continuous tracking. What I have today is a continuous track where if I display everything you get a rapidly changing heart rate. Not ideal. Or I grab the last item and get the same value every time. Also not ideal. Hoping to play with it more as I start to get some downtime at work again especially as my work project winds down.

Bad news for you Android users out there that MIGHT want to even just play with this, I've decided to focus entirely on getting up through the Pi integration on just iOS for awhile so that I can work out any issues on a tech stack that I'm most experienced in and what phone I primarily use day to day. Afterwards I promise to bring Android back up to parity with iOS. In fact remind me if I don't.

Video of current SPIKE behavior

*Update:
Decided to run the app for significantly longer and see if it ever updated the heart rate. It does, but at about 10minute intervals. More research into HealthKit and I learned that if I rely solely on HealthKit's information without a custom watchOS component 10min is about how long it will take to see any updates to the heart rate and/or step counters. Guess I'm building a watchOS app too with a minimal UI to just communicate back to the iOS app.

In other news my work project is officially over after today, and I'll be heading back to the bench likely for the remainder of the year. Hoping to do self-learning/content creation rather than a bench team, and if that works out should start seeing more progress updates on this project.

My goal is to have communication with a Pi turning on/off fans and/or lights by the end of the year.

 

[Fallen]

Update 3
It's been awhile, but time for another update. Spent more time reworking some things around how I'm using HealthKit for heart rate before moving on to other health data tracking. Finally made a breakthrough on Monday and got steps added and working today.

The rest of the health tracking info should be easier to add now that I've reorganized that section of code, so I've shifted my focus to getting basic health tracking completed first. At that point I'll get the iOS app submitted for public testing and set up a page to get access to it. Following the iOS health tracking completion I plan to switch back to Android and get it working for that so that I can get an alpha out for it too.

What's remaining on health tracking is likely to view how long the user has been standing, which will likely be more of a how long has the individual been standing of the last hour or other timeframe unless there's something within HealthKit that I don't yet know about for another idea. I'd also like to add a super simple tracker/reminder to drink water considering that's something that I way too often forget to do while at conventions. Apple's HealthKit provides a bunch of information that an app can request access to, so it's just a matter of figuring out what is most useful of that data and how to make it easy to view/access while maintaining that I want all display items to be opt-in and the app to work even if the user were to never enable displaying that data.


Attention Android Users:
If you have a wearable that you use to track health/fitness data? What do you use, and do you sync your data to Google Fit or use whatever company's app (assuming they provide one)?

 

[UNSC Marine Lt]

I love it!

 

[FalseShepherd]

Attention Android Users:
If you have a wearable that you use to track health/fitness data? What do you use, and do you sync your data to Google Fit or use whatever company's app (assuming they provide one)?

I use a samsung galaxy watch and sync to google fit and samsung health (google fit being the better of the two oddly enough). I don't know if I will wear it a cons - it might not fit with my forearm armor.
I have a google pixel 3 btw

 

[Fallen]

Minor update:
I took a few weeks off entirely to work on some other projects and focus on some things I've been wanting to learn or brush up on for awhile now. Remodeled my office and reworked my desk setup to make it easier for me to work. Not that it was bad before, but I'm always looking to improve things.



When I finally got back to the app, I started figuring out how appleStandTime is recorded in HealthKit. If I'm up walking around and moving my arms a fair amount it's more likely to be registered as stand time vs. standing but not moving around a lot. So if I were to be in a group at a convention that was stopped for any decent amount of time for photos and basically stood posing little to none of that time would actually be recorded as "standing time." I'm still debating the value in using it based on this info and might at least change it from sending a local notification to maybe take a break and just to an on screen value like HR and steps. For now I went ahead and added some device info in an easy to see spot such as battery and network info.

Finally purchased another Raspberry Pi that I'll use to set up that communication since my others have been repurposed into OctoPi's, so solving communication between the two and controlling a simple like circuit is probably next. My hope is to connect the two devices using BLE (Pi's are not part of the MFi program with Apple last I checked) rather than connecting to each other via WiFi in any way, but that also limits me on how much data I can send per "message." Still need to look into limitations.

 

[MrJamin]

Dang Fallen - I love the look in your office. I have to ask why the Mangler is upside down though :-D

 

[Fallen]

Thanks.

I have to ask why the Mangler is upside down though :-D

When my office chair is in that little nook, the wheels barely fit. I knocked the wall or drawer unit hard enough several times sliding the chair there that when the Mangler was standing upright it would fall off. Finally one of the blades broke off, so I turned it upside down after repairing until I get around to constructing a stand for it. Now to actually make said stand.

 

[Fallen]

It's hard to classify when something is a "minor update" from the outside perspective but took a lot of research and learning to get things set up and coded, and this is one of those updates.

I got another Pi so that I don't have to steal one of my OctoPi's for this project, and started getting it set up. I'm also dabbling with an Arduino and whichever ends up being simpler to start with I will continue on that front.

First step was to just have both able to run a script that turned on/off an LED. That wasn't challenging at all. Originally I was thinking I might run Swift on the Pi and the Arduino (possible but not well documented so nah). But I have another idea I've been playing with for the actual application implementation that makes Swift not the ideal choice for coding on either the Pi or Arduino. There is another language that I still might choose to work with on the devices, but for now I'm shelving that idea because it's easy enough to code in their typical languages. Plus that would make it easier to get answers to questions and/or bring others into the project if so desired.

A few of us at work were asked to look into Bluetooth Low Energy (BLE) for mobile development, so I looked into that from the iOS side and got as far as having my app be able to scan for a Pi broadcasting and connect to it. I didn't get any further than that before we were told a team was going to be spinning up to look into it. This frees me up to get back to what I really want to get done at this moment, sharing code between iOS and Android.

Marking as spoiler because attempted that path but ultimately chose not to keep it.

Warning: some technical jargon and nerding out over programming languages/features ahead

The more major shift that's occurring has to do with the language(s) that the apps are written in themselves. Up to this point I'd been writing everything for iOS as a Native iOS app written entirely in Swift. Android has been a Native Android app written using Kotlin. The downside to this is that for every change I make in iOS I have to write the code twice to also have the functionality in Android. While this is great for being able to interact with libraries provided by Apple and Google, it's duplicate work and code that slows down my overall development time.

So I finally am bringing in Kotlin Multiplatform. I'd briefly mentioned I hoped to go down that route in my first post, but now I've been learning enough to be able to start. The idea is that all of the views for the user interfaces will still be written natively, giving me power and access to still have a "native" feel where necessary, but all of the logic up to actually displaying information will be handled in a shared codebase written in Kotlin.

I'm actually pretty excited to be looking into this more as I'm hoping it's a promising way to get more iOS developers on board with cross-platform by being able to use it where it makes sense and stay in their native coding language/environment where it doesn't.

End nerding out. Tried to keep it short.

So far on that front I have brought Kotlin Multiplatform into my codebase, verified that the shared code is there and working properly, and am started implementing the settings and health data services (heart rate, steps, etc.) on the Android side while moving what I can from the iOS codebase to be in this shared business logic. It means no big visual changes on iOS and only a pre-step on Android, but I'm hopeful this will make further updates more streamlined.

 

[Fallen]

Update 4
It's been awhile since I've had something worth updating. I got pulled onto a research project where I actually learned a lot that's going to be useful for this project. I can't talk too much about what we were working on, but I can say that it meant working out how to work with Bluetooth Low Energy (BLE) on iOS and Android. Since I want this app to connect to a Pi and/or Arduino via BLE what I learned from that project will help immensely with that portion of this project. Too bad I also got pulled to another project just as I was going to learn integrating with Google Fit (since I already played with HealthKit on this project). Also eventually switched back out of KMM and instead implemented a procedural rule for the project to keep both platform in sync. Due to current use cases I wasn't getting enough of a benefit out of it as I'd hoped. Oh well. No code on either side got wasted so at least that's a positive.

Anyways, after going back and forth on it for awhile I've decided to go ahead and work on getting what I have released out for internal testing now rather than waiting even longer. Currently waiting for verification and approval from both Apple and Google for their developer programs and then should be able to push out some betas.

I'll update when that happens and will figure out how best to get individuals who want to test it out access. Internal testing has a limited number of openings and might include some regional limitations for testers as well, but I can't say definitively what it will look like as of yet.

So just to let you all know this project isn't dead, just a victim of life and scheduling.

That said I'm taking suggestions on potential app names and icons which will be particularly important if I attempt to make it publicly available on both stores.

Update 6/2
Temporarily removed features that iOS has completed but Android doesn't before uploading for testing.

Everything has been submitted for beta testing. Both Apple and Google have a review process for external testing, so just waiting for those reviews to go through before I can invite people to join testing.

Update 6/9
Apple reviewed and approved for external testing within a couple of days. Google just entered "In Review" status today for external testing. Will post links for those interested in joining testing whenever approval goes through. I'll post in what countries it's available at that same time.

 

[Fallen]

Well it certainly took awhile. After uploading a temporary app icon Google approved the app for open testing. Adding the links for anyone who may be interested in trying it out and staying updated with newest test releases.

iOS - will require to install TestFlight from Apple
Join iOS Beta

Android
Join Beta Group (Paused)

Release Notes:

• Basic UI is complete
• User can tap on buttons in the top left corner to play short audio clips

Next Steps:
I made the health data features on iOS unavailable because they haven't been implemented on Android yet. Getting heart rate and step counts as well as some helpful device information (e.g. battery level) is up next.

 

[EddieLadson]

Oh man I have been waiting for this I just downloaded the android beta. This is awesome I cant wait to get my armor done so i can implement this I have an old phone that I might install permanently this is so great.