Commit graph

346 commits

Author SHA1 Message Date
c89677abd7
dataset: explicit reshape 2022-09-02 16:57:59 +01:00
c066ea331b
dataset: namespace → dict
Python is so annoying
2022-09-02 16:07:44 +01:00
dbff15d4a9
add bug comment 2022-09-01 19:05:50 +01:00
9edda1f397
rainfallwrangler json2tfrecord.py: normalise data 2022-09-01 19:03:15 +01:00
3e4128c0a8
resize rainfall to be 1/2 size of current 2022-09-01 18:47:07 +01:00
8a86728b54
ai: how did I miss that 2022-09-01 18:07:53 +01:00
b2f1ba29bb
debug a 2022-09-01 18:05:33 +01:00
e2f621e212
even moar debug logging 2022-09-01 17:49:51 +01:00
2663123407
moar logging 2022-09-01 17:40:23 +01:00
c2fcb3b954
ai: channels_first → channels_last 2022-09-01 17:06:18 +01:00
f1d7973f22
ai: add dummy label 2022-09-01 17:01:00 +01:00
17d42fe899
ai: json.dumps 2022-09-01 16:21:52 +01:00
940f7aa1b5
ai: set self.model 2022-09-01 16:20:23 +01:00
f1be5fe2bd
ai: add missing arguments to LossContrastive 2022-09-01 16:14:00 +01:00
ddb375e906
ai: another typo 2022-09-01 16:12:05 +01:00
cb3e1d3a23
ai: fixup 2022-09-01 16:10:50 +01:00
e4c95bc7e3
typo 2022-09-01 16:09:24 +01:00
cfbbe8e8cf
ai: global? really? 2022-09-01 16:06:24 +01:00
4952ead094
ai: try the nonlocal keyword? I'm unsure what's going on here
....clearly I need to read up on scoping in python.
2022-09-01 16:02:37 +01:00
1d9d0fbb73
requirements.txt:a dd tensorflow_addons 2022-09-01 15:56:02 +01:00
8bdded23eb
ai: fix 'nother crash' name ConvNeXt submodels 2022-08-31 18:57:27 +01:00
b2a320134e
ai: typo 2022-08-31 18:54:03 +01:00
e4edc68df5
ai: add missing gamma layer 2022-08-31 18:52:35 +01:00
51cf08a386
ResNetRSV2 → ConvNeXt
ironically this makes the model simpler o/
2022-08-31 18:51:01 +01:00
3d614d105b
channels first 2022-08-31 18:06:59 +01:00
654eefd9ca
properly handle water dimensions; add log files to .gitignore
TODO: add heightmap
2022-08-31 18:03:39 +01:00
5846828f9e
debug logging 2022-08-31 17:48:09 +01:00
12c77e128d
handle feature_dim properly 2022-08-31 17:41:51 +01:00
c52a9f961c
and another 2022-08-31 17:37:28 +01:00
58dbabd561
fix another crash 2022-08-31 17:33:07 +01:00
5fc0725917
slurm: chmod +x 2022-08-31 16:33:07 +01:00
dbf929325a
typo; add pretrain slurm job file 2022-08-31 16:32:17 +01:00
e0162bc70b
requirements.txt: add missing dependencies 2022-08-31 16:25:47 +01:00
f2312c1184
fix crash 2022-08-31 16:25:27 +01:00
fe7a8b3fc0
Add 9 simple steps to use the rainfall radar model..... oh boy. 2022-08-31 16:18:28 +01:00
15a3519107
ai: the best thing about implementing a model is that you don't have to test it on the same day :P 2022-08-11 18:26:28 +01:00
28bcdf2192
vscode settings file 2022-08-11 16:38:25 +01:00
c0a9cb12d8
ai: start creating initial model implementation.
it's not hooked up to the CLI yet though.
Focus is still on ensuring the dataset is in the right format though
2022-08-10 19:03:25 +01:00
6cdf2b2389
wrangler python child: explicitly close stdout+stderr.
Hopefully this will avoid any more hanging issues.
2022-08-10 18:51:30 +01:00
5880bf9020
wrangler: add current date to process indicator.
There's a bug that causes it to hang, but we don't know why
2022-08-10 18:50:57 +01:00
231c832888
wrangler bugfix: crashes; logging output 2022-08-10 17:33:10 +01:00
b52c7f89a7
Move dataset parsing function to the right place 2022-08-10 17:24:55 +01:00
50f214450f
wrangler: fix crash 2022-08-10 17:05:01 +01:00
0bac8c8c0c
fixup 2022-08-08 17:23:24 +01:00
405f1a0bb0
fixup 2022-08-08 17:22:31 +01:00
5e1356513c
slurm: use compute, because 28 tf processes in parallel is too much for the GPU memory 2022-08-08 17:22:18 +01:00
133ef59af3
fixup 2022-08-08 16:33:05 +01:00
80e1a33ee2
slurm-jsonl2tfrecord.job: auto install dependencies 2022-08-08 16:31:49 +01:00
1442d20524
slurm: request gpu 2022-08-08 15:56:46 +01:00
f6f2e3694c
json2tfrecord: write slurm job file 2022-08-08 15:53:32 +01:00