Commit graph

366 commits

Author SHA1 Message Date
62f6a993bb
implement CBAM, but it's UNTESTED
Convolutional Block Attention Module.
2022-12-02 17:17:45 +00:00
9d666c3b38
train mono: type=int → float 2022-12-01 15:39:44 +00:00
53dfa32685
model_mono: log learning rate 2022-12-01 15:10:51 +00:00
c384d55dff
add arg to adjust learning rate 2022-11-29 20:55:00 +00:00
8e23e9d341
model_segmenter: we're no longer using sparse 2022-11-29 19:28:27 +00:00
9a2b4c6838
dsseg: fix reshape/onehot ordering 2022-11-29 19:28:13 +00:00
df774146d9
dataset_segmenter: reshape, not squeeze 2022-11-29 19:24:54 +00:00
77b8a1a8db
dataset_segmenter: squeeze 2022-11-29 19:16:15 +00:00
2258b5a229
slurm-train: reduce RAM required by 10GB 2022-11-29 19:15:34 +00:00
01101ad30b
losscrossentropy: return the reduced value * facepalm * 2022-11-29 19:07:08 +00:00
ff65393e78
log file naming update 2022-11-29 18:41:14 +00:00
37f196a785
LossCrossentropy: add kwargs 2022-11-29 15:40:35 +00:00
838ff56a3b
mono: fix loading checkpoint 2022-11-29 15:25:11 +00:00
dba6cbffcd
WHY. * facepalms * 2022-11-28 19:33:42 +00:00
57b8eb93fb
fixup 2022-11-28 19:09:35 +00:00
6640a41bb7
almost got it....? it's not what I expected....! 2022-11-28 19:08:50 +00:00
f48473b703
fixup 2022-11-28 19:00:11 +00:00
f6feb125e3
this iss ome serious debugging.
This commit will produce an extremely large volume of output.
2022-11-28 18:57:41 +00:00
09f81b0746
train_mono: debug
this commit will generate a large amount of debug output.
2022-11-28 16:46:17 +00:00
f39e4ade70
LayerConvNextGamma: fix config serialisation bug
.....this is unlikely to be the problem as this bug is in an unused code path.
2022-11-25 21:16:31 +00:00
e7410fb480
train_mono_predict: limit label size to 64x64
that's the size the model predicts
2022-11-25 17:47:17 +00:00
51dd484d13
fixup 2022-11-25 16:55:45 +00:00
884c4eb150
rainfall_stats: formatting again 2022-11-24 19:08:07 +00:00
bfe038086c
rainfall_stats: formatting 2022-11-24 19:07:44 +00:00
7dba03200f
fixup 2022-11-24 19:06:48 +00:00
e5258b9c66
typo 2022-11-24 19:06:13 +00:00
64d646bb13
rainfall_stats: formatting 2022-11-24 19:05:35 +00:00
675c7a7448
fixup 2022-11-24 19:03:28 +00:00
afc1cdcf02
fixup 2022-11-24 19:02:58 +00:00
e4bea89c89
typo 2022-11-24 19:01:52 +00:00
a40cbe8705
rainfall_stats: remove unused imports 2022-11-24 19:01:18 +00:00
fe57d6aab2
rainfall_stats: initial implementation
this might reveal why we are having problems. If most/all the rainfall radar
data is v small numbers, normalising
might help.
2022-11-24 18:58:16 +00:00
3131b4f7b3
debug2 2022-11-24 18:25:32 +00:00
d55a13f536
debug 2022-11-24 18:24:03 +00:00
1f60f2a580
do_argmax 2022-11-24 18:11:03 +00:00
6c09d5254d
fixup 2022-11-24 17:57:48 +00:00
54a841efe9
train_mono_predict: convert to correct format 2022-11-24 17:56:07 +00:00
105dc5bc56
missing kwargs 2022-11-24 17:51:29 +00:00
1e1d6dd273
fixup 2022-11-24 17:48:19 +00:00
011e0aef78
update cli docs 2022-11-24 16:38:07 +00:00
773944f9fa
train_mono_predict: initial implementation 2022-11-24 16:33:50 +00:00
d31326cb30
slurm train mono: fix partition name 2022-11-22 17:02:02 +00:00
ce28ac4013
slurm: add job for train_mono 2022-11-22 16:58:46 +00:00
3a0356929c
mono: drop the sparse 2022-11-22 16:20:56 +00:00
7e8f63f8ba
fixup 2022-11-21 19:38:24 +00:00
ace4c8b246
dataset_mono: debug 2022-11-21 18:46:21 +00:00
527b34942d
convnext_inverse: kernel_size 4→2 2022-11-11 19:29:37 +00:00
0662d0854b
model_mono: fix bottleneck 2022-11-11 19:11:40 +00:00
73acda6d9a
fix debug logging 2022-11-11 19:08:38 +00:00
9da059d738
model shape logging 2022-11-11 19:03:37 +00:00
00917b2698
dataset_mono: log shapes 2022-11-11 19:02:43 +00:00
54ae88b1b4
in this entire blasted project I have yet to get the rotation of anything correct....! 2022-11-11 18:58:45 +00:00
a7a475dcd1
debug 2 2022-11-11 18:38:07 +00:00
bf2f6e9b64
debug logging
it begins again
2022-11-11 18:31:40 +00:00
481eeb3759
mono: fix dataset preprocessing
rogue dimension
2022-11-11 18:31:27 +00:00
9035450213
mono: instantiate right model 2022-11-11 18:28:29 +00:00
69a2d0cf04
fixup 2022-11-11 18:27:01 +00:00
65e801cf28
train_mono: fix crash 2022-11-11 18:26:25 +00:00
8ac5159adc
dataset_mono: simplify param passing, onehot+threshold water depth data 2022-11-11 18:23:50 +00:00
3a3f7e85da
typo 2022-11-11 18:03:09 +00:00
3313f77c88
Add (untested) mono rainfall → water depth model
* sighs *
Unfortunately I can't seem to get contrastive learning to work.....
2022-11-10 22:36:11 +00:00
ce194d9227
slurm: customise log file names 2022-11-10 21:09:34 +00:00
9384b89165
model_segmentation: spare → normal crossentropy, activation functions at end 2022-11-10 20:53:37 +00:00
b6676e7361
switch from sparse to normal crossentropy 2022-11-10 20:50:56 +00:00
d8be26d476
Merge branch 'main' of git.starbeamrainbowlabs.com:sbrl/PhD-Rainfall-Radar 2022-11-10 20:49:01 +00:00
b03388de60
dataset_segmenter: DEBUG: fix water shape 2022-11-10 20:48:21 +00:00
daf691bf43
typo 2022-11-10 19:55:00 +00:00
0aa2ce19f5
read_metadata: support file inputs as well as dirs 2022-11-10 19:53:30 +00:00
aa7d9b8cf6
fixup 2022-11-10 19:46:09 +00:00
0894bd09e8
train_predict: add error message for parrams.json not found 2022-11-10 19:45:41 +00:00
0353072d15
allow pretrain to run on gpu
we've slashed the size of the 2nd encoder, so ti should fit naow?
2022-11-04 17:02:07 +00:00
44ad51f483
CallbackNBatchCsv: bugfix .sort() → sorted() 2022-11-04 16:40:21 +00:00
4dddcfcb42
pretrain_predict: missing \n 2022-11-04 16:01:28 +00:00
1375201c5f
CallbackNBatchCsv: open_handle mode 2022-11-03 18:29:00 +00:00
3206d6b7e7
slurm: rename segmenter job name 2022-11-03 17:12:27 +00:00
f2ae74ce7b
how could I be so stupid..... round 2 2022-11-02 17:38:26 +00:00
5f8d6dc6ea
Add metrics every 64 batches
this is important, because with large batches it can be difficult to tell what's happening inside each epoch.
2022-10-31 19:26:10 +00:00
cf872ef739
how could I be so *stupid*...... 2022-10-31 18:40:58 +00:00
da32d75778
make_callbacks: display steps, not samples 2022-10-31 18:36:28 +00:00
dfef7db421
moar debugging 2022-10-31 18:26:34 +00:00
172cf9d8ce
tweak 2022-10-31 18:19:43 +00:00
dbe35ee943
loss: comment l2 norm 2022-10-31 18:09:03 +00:00
5e60319024
fixup 2022-10-31 17:56:49 +00:00
b986b069e2
debug party time 2022-10-31 17:50:29 +00:00
458faa96d2
loss: fixup 2022-10-31 17:18:21 +00:00
55dc05e8ce
contrastive: comment weights that aren't needed 2022-10-31 16:26:48 +00:00
33391eaf16
train_predict/jsonl: don't argmax
I'm interested inthe raw values
2022-10-26 17:21:19 +01:00
74f2cdb900
train_predict: .list() → .tolist() 2022-10-26 17:12:36 +01:00
4f9d543695
train_predict: don't pass model_code
it's redundant
2022-10-26 17:11:36 +01:00
1b489518d0
segmenter: add LayerStack2Image to custom_objects 2022-10-26 17:05:50 +01:00
48ae8a5c20
LossContrastive: normalise features as per the paper 2022-10-26 16:52:56 +01:00
843cc8dc7b
contrastive: rewrite the loss function.
The CLIP paper *does* kinda make sense I think
2022-10-26 16:45:45 +01:00
fad1399c2d
convnext: whitespace 2022-10-26 16:45:20 +01:00
1d872cb962
contrastive: fix initial temperature value
It should be 1/0.07, but we had it set to 0.07......
2022-10-26 16:45:01 +01:00
f994d449f1
Layer2Image: fix 2022-10-25 21:32:17 +01:00
6a29105f56
model_segmentation: stack not reshape 2022-10-25 21:25:15 +01:00
98417a3e06
prepare for NCE loss
.....but Tensorflow's implementation looks to be for supervised models :-(
2022-10-25 21:15:05 +01:00
bb0679a509
model_segmentation: don't softmax twice 2022-10-25 21:11:48 +01:00
f2e2ca1484
model_contrastive: make water encoder significantly shallower 2022-10-24 20:52:31 +01:00
a6b07a49cb
count water/nowater pixels in Jupyter Notebook 2022-10-24 18:05:34 +01:00