Commit graph

219 commits

Author SHA1 Message Date
59cfa4a89a
basename paths 2022-10-20 15:11:14 +01:00
200076596b
finish train_predict 2022-10-19 17:26:40 +01:00
63e909d9fc
datasets: add shuffle=True/False to get_filepaths.
This is important because otherwise it SCAMBLES the filenames, which is a disaster for making predictions in the right order....!
2022-10-19 16:52:07 +01:00
fe43ddfbf9
start implementing driver for train_predict, but not finished yet 2022-10-18 19:37:55 +01:00
b3ea189d37
segmentation: softmax the output 2022-10-13 21:02:57 +01:00
f121bfb981
fixup summaryfile 2022-10-13 17:54:42 +01:00
5c35c0cee4
model_segmentation: document; remove unused args 2022-10-13 17:50:16 +01:00
f12e6ab905
No need for a CLI arg for feature_dim_in - metadata should contain this 2022-10-13 17:37:16 +01:00
ae53130e66
layout 2022-10-13 14:54:20 +01:00
6423bf6702
LayerConvNeXtGamma: avoid adding an EagerTensor to config
Very weird how this is a problem when it wasn't before..
2022-10-12 17:12:07 +01:00
32f5200d3b
pass model_arch properly 2022-10-12 16:50:06 +01:00
c45b90764e
segmentation: adds xxtiny, but unsure if it's small enough 2022-10-11 19:22:37 +01:00
11f91a7cf4
train: add --arch; default to convnext_i_xtiny 2022-10-11 19:18:01 +01:00
e9a8e2eb57
fixup 2022-10-06 19:23:31 +01:00
9f3ae96894
finish wiring for --water-size 2022-10-06 19:21:50 +01:00
5dac70aa08
typo 2022-10-06 19:17:03 +01:00
2960d3b645
exception → warning 2022-10-06 18:26:40 +01:00
0ee6703c1e
Add todo and comment 2022-10-03 19:06:56 +01:00
2b182214ea
typo 2022-10-03 17:53:10 +01:00
92c380bff5
fiddle with Conv2DTranspose
you need to set the `stride` argument to actually get it to upscale..... :P
2022-10-03 17:51:41 +01:00
d544553800
fixup 2022-10-03 17:33:06 +01:00
058e3b6248
model_segmentation: cast float → int 2022-10-03 17:31:36 +01:00
04e5ae0c45
model_segmentation: redo reshape
much cheese was applied :P
2022-10-03 17:27:52 +01:00
deffe69202
typo 2022-10-03 16:59:36 +01:00
fc6d2dabc9
Upscale first, THEN convnext... 2022-10-03 16:38:43 +01:00
6a0790ff50
convnext_inverse: add returns; change ordering 2022-10-03 16:32:09 +01:00
e51087d0a9
add reshape layer 2022-09-28 18:22:48 +01:00
a336cdee90
and continues 2022-09-28 18:18:10 +01:00
de47a883d9
missing units 2022-09-28 18:17:22 +01:00
b5e08f92fe
the long night continues 2022-09-28 18:14:09 +01:00
dc159ecfdb
and again 2022-09-28 18:11:46 +01:00
4cf0485e32
fixup... again 2022-09-28 18:10:11 +01:00
030d8710b6
fixup 2022-09-28 18:08:31 +01:00
4ee7f2a0d6
add water thresholding 2022-09-28 18:07:26 +01:00
41ba980d69
segmentationP implement dataset parser 2022-09-28 17:19:21 +01:00
e9e6139c7a
typo 2022-09-28 16:28:18 +01:00
d6ff3fb2ce
pretrain_predict fix write mode 2022-09-27 17:38:12 +01:00
f95fd8f9e4
pretrain-predict: add .tfrecord output function 2022-09-27 16:59:31 +01:00
30b8dd063e
fixup 2022-09-27 15:54:37 +01:00
3cf99587e4
Contraster: add gamma layer to load_model 2022-09-27 15:53:52 +01:00
d59de41ebb
embeddings: change title rendering; make even moar widererer
We need to see that parallel coordinates  plot in detail
2022-09-23 18:56:39 +01:00
5252a81238
vis: don't call add_subplot 2022-09-20 19:06:21 +01:00
a552cc4dad
ai vis: make parallel coordinates wider 2022-09-16 18:51:49 +01:00
a70794e661
umap: no min_dist 2022-09-16 17:09:09 +01:00
5778fc51f7
embeddings: fix title; remove colourmap 2022-09-16 17:08:04 +01:00
fcab227f6a
cheese: set label for everything to 1 2022-09-16 16:42:05 +01:00
1e35802d2b
ai: fix embed i/o 2022-09-16 16:02:27 +01:00
ed94da7492
fixup 2022-09-16 15:51:26 +01:00
366db658a8
ds predict: fix filenames in 2022-09-16 15:45:22 +01:00
e333dcba9c
tweak projection head 2022-09-16 15:36:01 +01:00
6defd24000
bugfix: too many values to unpack 2022-09-15 19:56:17 +01:00
e3c8277255
ai: tweak the segmentation model structure 2022-09-15 19:54:50 +01:00
bd64986332
ai: implement batched_iterator to replace .batch()
...apparently .batch() means you get a BatchedDataset or whatever when you iterate it like a tf.function instead of the actual tensor :-/
2022-09-15 19:16:38 +01:00
ccd256c00a
embed rainfall radar, not both 2022-09-15 17:37:04 +01:00
2c74676902
predict → predict_on_batch 2022-09-15 17:31:50 +01:00
f036e79098
fixup 2022-09-15 17:09:26 +01:00
d5f1a26ba3
disable prefetching when predicting a thing 2022-09-15 17:09:09 +01:00
8770638022
ai: call .predict(), not the model itself 2022-09-14 17:41:01 +01:00
a96cefde62
ai: predict oops 2022-09-14 17:37:48 +01:00
fa3165a5b2
dataset: simplify dataset_predict 2022-09-14 17:33:17 +01:00
279e27c898
fixup 2022-09-14 17:16:49 +01:00
fad3313ede
fixup 2022-09-14 17:14:04 +01:00
1e682661db
ai: kwargs in from_checkpoint 2022-09-14 17:11:06 +01:00
6bda24d4da
ai: how did I miss that?!
bugfix ah/c
2022-09-14 16:53:43 +01:00
decdd434d8
ai from_checkpoint: bugfix 2022-09-14 16:49:01 +01:00
7685ec3e8b
implement ability to embed & plot pretrained embeddings 2022-09-13 19:18:59 +01:00
7130c4fdf8
start implementing core image segmentation model 2022-09-07 17:45:38 +01:00
22620a1854
ai: implement saving only the rainfall encoder 2022-09-06 19:48:46 +01:00
3e13ad12c8
ai Bugfix LayerContrastiveEncoder: channels → input_channels
for consistency
2022-09-05 23:53:16 +01:00
9d39215dd5
dataset: drop incomplete batches 2022-09-05 15:36:10 +01:00
1d1533d160
ai: how did things get this confusing 2022-09-02 18:51:46 +01:00
1c5defdcd6
ai: cats 2022-09-02 18:45:23 +01:00
6135bcd0cd
fixup 2022-09-02 18:41:31 +01:00
c33c0a0899
ai: these shapes are so annoying 2022-09-02 18:39:24 +01:00
23bcd294c4
AI: 0.75 → 0.5? 2022-09-02 18:12:51 +01:00
ad156a9a00
ai dataset: centre crop the water data to 75% original size
this should both help the model and reduce memory usage
2022-09-02 18:05:32 +01:00
efe41b96ec
fixup 2022-09-02 17:58:02 +01:00
f8ee0afca1
ai: fix arch_name plumbing 2022-09-02 17:57:07 +01:00
3d44831080
Add NO_PREFETCH env var 2022-09-02 17:55:04 +01:00
3e0ca6a315
ai: fix summary file writing; make water encoder smaller 2022-09-02 17:51:45 +01:00
389216b391
reordering 2022-09-02 17:31:19 +01:00
b9d01ddadc
summary logger → summarywriter 2022-09-02 17:28:00 +01:00
9f7f4af784
dataset: properly resize 2022-09-02 17:09:38 +01:00
c89677abd7
dataset: explicit reshape 2022-09-02 16:57:59 +01:00
c066ea331b
dataset: namespace → dict
Python is so annoying
2022-09-02 16:07:44 +01:00
dbff15d4a9
add bug comment 2022-09-01 19:05:50 +01:00
3e4128c0a8
resize rainfall to be 1/2 size of current 2022-09-01 18:47:07 +01:00
8a86728b54
ai: how did I miss that 2022-09-01 18:07:53 +01:00
b2f1ba29bb
debug a 2022-09-01 18:05:33 +01:00
e2f621e212
even moar debug logging 2022-09-01 17:49:51 +01:00
2663123407
moar logging 2022-09-01 17:40:23 +01:00
c2fcb3b954
ai: channels_first → channels_last 2022-09-01 17:06:18 +01:00
f1d7973f22
ai: add dummy label 2022-09-01 17:01:00 +01:00
17d42fe899
ai: json.dumps 2022-09-01 16:21:52 +01:00
940f7aa1b5
ai: set self.model 2022-09-01 16:20:23 +01:00
f1be5fe2bd
ai: add missing arguments to LossContrastive 2022-09-01 16:14:00 +01:00
ddb375e906
ai: another typo 2022-09-01 16:12:05 +01:00
cb3e1d3a23
ai: fixup 2022-09-01 16:10:50 +01:00
e4c95bc7e3
typo 2022-09-01 16:09:24 +01:00
cfbbe8e8cf
ai: global? really? 2022-09-01 16:06:24 +01:00
4952ead094
ai: try the nonlocal keyword? I'm unsure what's going on here
....clearly I need to read up on scoping in python.
2022-09-01 16:02:37 +01:00
8bdded23eb
ai: fix 'nother crash' name ConvNeXt submodels 2022-08-31 18:57:27 +01:00
b2a320134e
ai: typo 2022-08-31 18:54:03 +01:00
e4edc68df5
ai: add missing gamma layer 2022-08-31 18:52:35 +01:00
51cf08a386
ResNetRSV2 → ConvNeXt
ironically this makes the model simpler o/
2022-08-31 18:51:01 +01:00
3d614d105b
channels first 2022-08-31 18:06:59 +01:00
654eefd9ca
properly handle water dimensions; add log files to .gitignore
TODO: add heightmap
2022-08-31 18:03:39 +01:00
5846828f9e
debug logging 2022-08-31 17:48:09 +01:00
12c77e128d
handle feature_dim properly 2022-08-31 17:41:51 +01:00
c52a9f961c
and another 2022-08-31 17:37:28 +01:00
58dbabd561
fix another crash 2022-08-31 17:33:07 +01:00
f2312c1184
fix crash 2022-08-31 16:25:27 +01:00
15a3519107
ai: the best thing about implementing a model is that you don't have to test it on the same day :P 2022-08-11 18:26:28 +01:00
c0a9cb12d8
ai: start creating initial model implementation.
it's not hooked up to the CLI yet though.
Focus is still on ensuring the dataset is in the right format though
2022-08-10 19:03:25 +01:00
b52c7f89a7
Move dataset parsing function to the right place 2022-08-10 17:24:55 +01:00
323d708692
dataset: add todo
just why, Tensorflow?!
tf.data.TextLineDataset looks almost too good to be true..... and it is, as despite supporting decompressing via gzip(!) it doesn't look like we can convince it to parse JSON :-/
2022-07-26 19:53:18 +01:00
e93a95f1b3
ai dataset: add if main == main 2022-07-26 19:24:40 +01:00
de4c3dab17
typo 2022-07-26 19:14:55 +01:00
18a7d3674b
ai: create (untested) dataset 2022-07-26 19:14:10 +01:00