Commit graph

566 commits

Author SHA1 Message Date
Starbeamrainbowlabs 920cc3feaf
Properly close last writer
otherwise Node.js doesn't quit
2022-07-04 17:04:11 +01:00
Starbeamrainbowlabs 588ee87b83
Bugfix: fix end-of-file 2022-07-01 19:34:26 +01:00
Starbeamrainbowlabs 5b2d71f41f
it works
.....I think
2022-07-01 19:08:36 +01:00
Starbeamrainbowlabs 1297f41105
.tfrecord files are too much hassle
let's go with a standard of .jsonl.gz instead
2022-07-01 18:28:39 +01:00
Starbeamrainbowlabs f5f267c6b6
Update dependencies 2022-07-01 16:56:51 +01:00
Starbeamrainbowlabs ba258fbba0
Remove debug loogging 2022-05-19 19:25:44 +01:00
Starbeamrainbowlabs e030e6c2d5
Fix remaining(?) crashes= in our code 2022-05-19 19:13:28 +01:00
Starbeamrainbowlabs 3cb7e42505
it doesn't crash as much now, but it still isn't behaving. 2022-05-19 18:52:15 +01:00
Starbeamrainbowlabs bb018c53f6
Fix many bugs
Many bugs remain though
2022-05-19 17:54:14 +01:00
Starbeamrainbowlabs cc5efbae8a
Implement tfrecodify subcommand.
It's all still untested, but that's the next step
2022-05-19 17:15:15 +01:00
Starbeamrainbowlabs 0fa7ae9d6a
Imnplement plumbing, but it's all untested 2022-05-18 17:47:02 +01:00
Starbeamrainbowlabs bf4866bdbc
Add data readers 2022-05-18 17:04:11 +01:00
Starbeamrainbowlabs 5829db062b
Merge branch 'main' of git.starbeamrainbowlabs.com:sbrl/PhD-Rainfall-Radar 2022-05-13 19:09:03 +01:00
Starbeamrainbowlabs 58dfaf2ece Initial commit 2022-05-13 19:08:14 +01:00
Starbeamrainbowlabs 9411ad3218
tweak licence 2022-05-13 19:08:04 +01:00
Starbeamrainbowlabs 8a9cd6c1c0
Lay out some basic scaffolding
I *really* hope this works. This is the 3rd major revision of this
model. I've learnt a ton of stuff between now and my last attempt, so
here's hoping that all goes well :D

The basic idea behind this attempt is *Contrastive Learning*. If we
don't get anything useful with this approach, then we can assume that
it's not really possible / feasible.

Something we need to watch out for is the variance (or rather lack
thereof) in the dataset. We have 1.5M timesteps, but not a whole lot
will be happening in most of those....

We may need to analyse the variance of the water depth data and extract
a subsample that's more balanced.
2022-05-13 19:06:15 +01:00