Resolved Program hanging when loading a large data set from QIF

[gotenks05]

Hey there, everyone. It's been a while since I been to this place, and I am here because I really need help with an issue that is giving me grief.

I am working on an update for a multi-platform app I made to keep track of finances that will allow it to import data from QIF, in addition to a custom JSON file, and while the iOS side does not seem to be presenting any problems, I am having troubles with the mac side of the equation, in which the program hangs when presented with a large data set.

From my observations and setting breakpoints setup in my project, the program is retrieving the data as expected and the data is stored in the SQLite database, as it should, regardless of the size of data, with no problems whatsoever when pushing example data that has been modified to remove the comments and formatting the date to MM/DD/YYYY format or even smaller sets.

However, when I try to test importing a QIF file generated by GnuCash to QIF formatted for Quicken 2003, the program hangs when it is time for the view to be refreshed or initially loaded, even though it managed to retrieve around 756 objects and successfully saved them to the database.

When I initially noticed the hangs, I tried implementing async functions, to try and move the time consuming processes to another thread, as well as to allow a way to give the user a signal that the program is attempting to load data, but the program still ends up hanging on that particular import.

How do I go about fixing this?

Here are the functions doing the work that is supposed to be refreshing the view:

func retrieveRecords() async -> [Record] {

        guard let databaseManager = DB.shared.manager, let storedRecords = try? databaseManager.records(inRange: .all) else { return [] }

        return storedRecords
 }

 func loadRecords() {

      Task {


          let storedRecords = await retrieveRecords()


           


          records.items = storedRecords


      }
}

For context, my project's code containing the functionality in question can be found at the link below:

GitHub - bryceac/BCheckbook at qifSupport

Simple Checkbook Ledger for iOS and macOS. Contribute to bryceac/BCheckbook development by creating an account on GitHub.

github.com

The mac code is in the macOS folder, the iOS folder contains the iOS code, and Shared contains shared elements.

If you need to see how I am parsing QIF files, the code for the custom library I wrote can be found at the link below:

GitHub - bryceac/QIF: library to read and write QIF files using Swift.

library to read and write QIF files using Swift. Contribute to bryceac/QIF development by creating an account on GitHub.

github.com

 

[gotenks05]

I think I may have found my issue, though I will likely keep this open a while longer.

When I tried looking into debugging the hang, by doing a pause, as suggested on this page, I found that a dictionary I was creating to store balance values may have been the cause and tried converting the dictionary to a function that overlays a call to the database.

While it still took a while to populate, my app did not hang.

Update: I did some more testing and it looks like I am still having issues. However, rather than both loading data from a file and launching the app after the import, it looks like the program still hangs when being launched after the import.

Sadly, the pause suggestion did not help out too much because it did not point towards any particular point, aside from showing up above the main marker in the mac source code and a breakpoint in the ContentView file in the macOS folder showed that things were being retrieved otherwise, so I'm glad I did not close this.

Any help would still be appreciated because this will definitely be a problem with the user experience if I do not fix it. The code available on Github should be up to date with I have now.

 

[gotenks05]

Ok, things are still not optimal, but I made the function I am using to retrieve items present the loading overlay I set up and I was able to verify the data is loading in a timely enough manner on both launch and after import (as long I present the user with the loading overlay), so it looks like this particular issue was indeed resolved by getting rid of the dictionary I used to try and limit database calls.

While I'm not exactly happy with my finding, I will be marking this solved.