No. Just no image projects for me. I don't have the time or computer power to figure this out.
Time to attempt a more classic problem that only uses a csv.
Friday, July 26, 2019
Monday, July 22, 2019
Capstone Adventures: Part 7
I have tensorflow! After wrangling with environment builds and variations of tensorflow and cuda, I finally find a stackoverflow post that says I can just install keras-gpu on anaconda.
Next batch of things to learn:
I am thankful for Keras-GPU and the guy who mentioned in on StackOverflow. You're my hero!
Next batch of things to learn:
- Use Tensorflow to train my data
- Dump the model into a pickle file
- Design an application that fits all capstone requirements
- Integrate the pickle in the application
I am thankful for Keras-GPU and the guy who mentioned in on StackOverflow. You're my hero!
Sunday, July 21, 2019
Capstone Adventures: Part 6
I think I've decided that I'm going for the Diabetic Retinopathy (leading cause of blindness) project. Getting this to work will involve either pushing my current system to run tensorflow-gpu or getting a new computer. I've been attempting to install it all day, going back and forth between CUDA9, CUDA10, and several variations of other parts of the installation.
After the first few failed attempts I figured that a new computer will likely not help and just realized at 11pm, that Tensorflow absolutely cannot work with Python 3.7. As you can clearly see ... I'm really not that smart ... This experience has been great for my self-esteem. To Python 3.6 and the glimmer of hope that I'm a little closer to figuring this out than I was 12 hours ago.
After the first few failed attempts I figured that a new computer will likely not help and just realized at 11pm, that Tensorflow absolutely cannot work with Python 3.7. As you can clearly see ... I'm really not that smart ... This experience has been great for my self-esteem. To Python 3.6 and the glimmer of hope that I'm a little closer to figuring this out than I was 12 hours ago.
Friday, July 19, 2019
Capstone Adventures: Part 5
Well, the data reappeared but then I deleted everything the next day after my bill spiked up to over $500. So there goes that idea.
Right now I'm bouncing between another image-based dataset with under 5k images and an e-commerce dataset for customer segmentation or a market basket analysis. I really want to do an image one because it's not something I could have done before BSCS and the other two I could have done after MSDA.
I'm exploring a few tools right now, trying to decide which one is more feasible. If I used a service like Clarifai, it would cut down on the number of things I need to build from scratch. I have another week before I really need to buckle down and get things moving.
Right now I'm bouncing between another image-based dataset with under 5k images and an e-commerce dataset for customer segmentation or a market basket analysis. I really want to do an image one because it's not something I could have done before BSCS and the other two I could have done after MSDA.
I'm exploring a few tools right now, trying to decide which one is more feasible. If I used a service like Clarifai, it would cut down on the number of things I need to build from scratch. I have another week before I really need to buckle down and get things moving.
Tuesday, July 16, 2019
Capstone Adventures: Part 4
I got through my dataset, had everything uploaded, then started training. My mistake was not knowing how much credit that was using though I supposed that's Microsoft Azure's mistake because it didn't tell me how much it would cost. So I pick the 24 hour option because why not? And I run out of credit which I thought might be a thing but it was the last part I needed before I could start programming it.
I upgrade my account and that is when this project died. It is no longer there. All my data and training is just gone.
Time for a new project.
I upgrade my account and that is when this project died. It is no longer there. All my data and training is just gone.
Time for a new project.
Sunday, July 14, 2019
Capstone Adventures: Part 3
Well, it looks like this dataset might be too big for Computer Vision. I just realized that it has a 100k upload limit which is less than half my training dataset.
EDIT:
... and my dataset is gone. Apparently I built the last project on my "student" account which just expired. I'm going to continue classifying my files since the remaining data will fill up the 100k limit easily. But I'll wait to hear from my advisor before recreating the project on the "free" account.
I also have another project lined up just in case I end up nixing this idea all together: Facial Expression Recognition. I would need to converting a .bib file into jpeg files to get it to work. But it's under 30k images so even my computer can handle this one.
RAWR!!!
On the upside, I am acquiring useful data munging and automation skills.
EDIT:
... and my dataset is gone. Apparently I built the last project on my "student" account which just expired. I'm going to continue classifying my files since the remaining data will fill up the 100k limit easily. But I'll wait to hear from my advisor before recreating the project on the "free" account.
I also have another project lined up just in case I end up nixing this idea all together: Facial Expression Recognition. I would need to converting a .bib file into jpeg files to get it to work. But it's under 30k images so even my computer can handle this one.
RAWR!!!
On the upside, I am acquiring useful data munging and automation skills.
Saturday, July 13, 2019
Capstone Adventures: Part 2
It took a few tweaks to get the image search and sort scripts working. The process is finally streamlined and I was able to get through 3 of the 14 batches until I stopped to attempt to send some of those files to Azure.
Of course this didn't work, otherwise I wouldn't have had to add an extra post today. Apparently Azure doesn't like .tif images. After confirming that it prefers some other file format, I installed IrfanView to start batch converting them before upload.
The last hiccough of the day was having to stop that process because my computer was running out of space again. So I'm now uploading the files into Azure so I can make room to convert more files. I think I'll finish uploading the files I processed so far today, then tomorrow I can sort, convert, and upload at the same time (in theory) ... get some sort of assembly line going, lol.
Adventure!
Of course this didn't work, otherwise I wouldn't have had to add an extra post today. Apparently Azure doesn't like .tif images. After confirming that it prefers some other file format, I installed IrfanView to start batch converting them before upload.
The last hiccough of the day was having to stop that process because my computer was running out of space again. So I'm now uploading the files into Azure so I can make room to convert more files. I think I'll finish uploading the files I processed so far today, then tomorrow I can sort, convert, and upload at the same time (in theory) ... get some sort of assembly line going, lol.
Adventure!
Subscribe to:
Posts (Atom)