sbrl-github/research-rainfallradar
1
0
Fork 0
mirror of https://github.com/sbrl/research-rainfallradar synced 2024-07-07 13:24:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
361 commits 1 branch 0 tags 2.1 MiB
Commit graph

258 commits

Author SHA1 Message Date
Starbeamrainbowlabs 5f8d6dc6ea
Add metrics every 64 batches 
this is important, because with large batches it can be difficult to tell what's happening inside each epoch.
 2022-10-31 19:26:10 +00:00
Starbeamrainbowlabs cf872ef739
how could I be so *stupid*...... 2022-10-31 18:40:58 +00:00
Starbeamrainbowlabs da32d75778
make_callbacks: display steps, not samples 2022-10-31 18:36:28 +00:00
Starbeamrainbowlabs dfef7db421
moar debugging 2022-10-31 18:26:34 +00:00
Starbeamrainbowlabs 172cf9d8ce
tweak 2022-10-31 18:19:43 +00:00
Starbeamrainbowlabs dbe35ee943
loss: comment l2 norm 2022-10-31 18:09:03 +00:00
Starbeamrainbowlabs 5e60319024
fixup 2022-10-31 17:56:49 +00:00
Starbeamrainbowlabs b986b069e2
debug party time 2022-10-31 17:50:29 +00:00
Starbeamrainbowlabs 458faa96d2
loss: fixup 2022-10-31 17:18:21 +00:00
Starbeamrainbowlabs 55dc05e8ce
contrastive: comment weights that aren't needed 2022-10-31 16:26:48 +00:00
Starbeamrainbowlabs 33391eaf16
train_predict/jsonl: don't argmax 
I'm interested inthe raw values
 2022-10-26 17:21:19 +01:00
Starbeamrainbowlabs 74f2cdb900
train_predict: .list() → .tolist() 2022-10-26 17:12:36 +01:00
Starbeamrainbowlabs 4f9d543695
train_predict: don't pass model_code 
it's redundant
 2022-10-26 17:11:36 +01:00
Starbeamrainbowlabs 1b489518d0
segmenter: add LayerStack2Image to custom_objects 2022-10-26 17:05:50 +01:00
Starbeamrainbowlabs 48ae8a5c20
LossContrastive: normalise features as per the paper 2022-10-26 16:52:56 +01:00
Starbeamrainbowlabs 843cc8dc7b
contrastive: rewrite the loss function. 
The CLIP paper *does* kinda make sense I think
 2022-10-26 16:45:45 +01:00
Starbeamrainbowlabs fad1399c2d
convnext: whitespace 2022-10-26 16:45:20 +01:00
Starbeamrainbowlabs 1d872cb962
contrastive: fix initial temperature value 
It should be 1/0.07, but we had it set to 0.07......
 2022-10-26 16:45:01 +01:00
Starbeamrainbowlabs f994d449f1
Layer2Image: fix 2022-10-25 21:32:17 +01:00
Starbeamrainbowlabs 6a29105f56
model_segmentation: stack not reshape 2022-10-25 21:25:15 +01:00
Starbeamrainbowlabs 98417a3e06
prepare for NCE loss 
.....but Tensorflow's implementation looks to be for supervised models :-(
 2022-10-25 21:15:05 +01:00
Starbeamrainbowlabs bb0679a509
model_segmentation: don't softmax twice 2022-10-25 21:11:48 +01:00
Starbeamrainbowlabs f2e2ca1484
model_contrastive: make water encoder significantly shallower 2022-10-24 20:52:31 +01:00
Starbeamrainbowlabs a6b07a49cb
count water/nowater pixels in Jupyter Notebook 2022-10-24 18:05:34 +01:00
Starbeamrainbowlabs a8b101bdae
dataset_predict: add shape_water_desired 2022-10-24 18:05:13 +01:00
Starbeamrainbowlabs 587c1dfafa
train_predict: revamp jsonl handling 2022-10-21 16:53:08 +01:00
Starbeamrainbowlabs 8195318a42
SparseCategoricalAccuracy: losses → metrics 2022-10-21 16:51:20 +01:00
Starbeamrainbowlabs 612735aaae
rename shuffle arg 2022-10-21 16:35:45 +01:00
Starbeamrainbowlabs c98d8d05dd
segmentation: use the right accuracy 2022-10-21 16:17:05 +01:00
Starbeamrainbowlabs bb0258f5cd
flip squeeze operator ordering 2022-10-21 15:38:57 +01:00
Starbeamrainbowlabs af26964c6a
batched_iterator: reset i_item after every time 2022-10-21 15:35:43 +01:00
Starbeamrainbowlabs c5b1501dba
train-predict fixup 2022-10-21 15:27:39 +01:00
Starbeamrainbowlabs 42aea7a0cc
plt.close() fixup 2022-10-21 15:23:54 +01:00
Starbeamrainbowlabs 12dad3bc87
vis/segmentation: fix titles 2022-10-21 15:22:35 +01:00
Starbeamrainbowlabs 0cb2de5d06
train-preedict: close matplotlib after we've finished 
they act like file handles
 2022-10-21 15:19:31 +01:00
Starbeamrainbowlabs 81e53efd9c
PNG: create output dir if doesn't exist 2022-10-21 15:17:39 +01:00
Starbeamrainbowlabs 3f7db6fa78
fix embedding confusion 2022-10-21 15:15:59 +01:00
Starbeamrainbowlabs 847cd97ec4
fixup 2022-10-21 14:26:58 +01:00
Starbeamrainbowlabs 0e814b7e98
Contraster → Segmenter 2022-10-21 14:25:43 +01:00
Starbeamrainbowlabs 1b658a1b7c
train-predict: can't destructure array when iterating generator 
....it seems to lead to undefined behaviour or something
 2022-10-20 19:34:04 +01:00
Starbeamrainbowlabs aed2348a95
train_predict: fixup 2022-10-20 15:42:33 +01:00
Starbeamrainbowlabs cc6679c609
batch data; use generator 2022-10-20 15:22:29 +01:00
Starbeamrainbowlabs d306853c42
use right daataset 2022-10-20 15:16:24 +01:00
Starbeamrainbowlabs 59cfa4a89a
basename paths 2022-10-20 15:11:14 +01:00
Starbeamrainbowlabs 4d8ae21a45
update cli help text 2022-10-19 17:31:42 +01:00
Starbeamrainbowlabs 200076596b
finish train_predict 2022-10-19 17:26:40 +01:00
Starbeamrainbowlabs 488f78fca5
pretrain_predict: default to parallel_reads=0 2022-10-19 16:59:45 +01:00
Starbeamrainbowlabs 63e909d9fc
datasets: add shuffle=True/False to get_filepaths. 
This is important because otherwise it SCAMBLES the filenames, which is a disaster for making predictions in the right order....!
 2022-10-19 16:52:07 +01:00
Starbeamrainbowlabs fe43ddfbf9
start implementing driver for train_predict, but not finished yet 2022-10-18 19:37:55 +01:00
Starbeamrainbowlabs b3ea189d37
segmentation: softmax the output 2022-10-13 21:02:57 +01:00
Starbeamrainbowlabs f121bfb981
fixup summaryfile 2022-10-13 17:54:42 +01:00
Starbeamrainbowlabs 5c35c0cee4
model_segmentation: document; remove unused args 2022-10-13 17:50:16 +01:00
Starbeamrainbowlabs f12e6ab905
No need for a CLI arg for feature_dim_in - metadata should contain this 2022-10-13 17:37:16 +01:00
Starbeamrainbowlabs e201372252
write quick Jupyter notebook to test data 
....I'm paranoid
 2022-10-13 17:27:17 +01:00
Starbeamrainbowlabs ae53130e66
layout 2022-10-13 14:54:20 +01:00
Starbeamrainbowlabs 6423bf6702
LayerConvNeXtGamma: avoid adding an EagerTensor to config 
Very weird how this is a problem when it wasn't before..
 2022-10-12 17:12:07 +01:00
Starbeamrainbowlabs 32f5200d3b
pass model_arch properly 2022-10-12 16:50:06 +01:00
Starbeamrainbowlabs 5933fb1061
fixup 2022-10-11 19:23:41 +01:00
Starbeamrainbowlabs c45b90764e
segmentation: adds xxtiny, but unsure if it's small enough 2022-10-11 19:22:37 +01:00
Starbeamrainbowlabs f4a2c742d9
typo 2022-10-11 19:19:23 +01:00
Starbeamrainbowlabs 11f91a7cf4
train: add --arch; default to convnext_i_xtiny 2022-10-11 19:18:01 +01:00
Starbeamrainbowlabs 5666c5a0d9
typo 2022-10-10 18:12:51 +01:00
Starbeamrainbowlabs 131c0a0a5b
pretrain-predict: create dir if not exists 2022-10-10 18:00:55 +01:00
Starbeamrainbowlabs f883986eaa
Bugfix: modeset to enable TFRecordWriter instead of bare handle 2022-10-06 20:07:59 +01:00
Starbeamrainbowlabs e9a8e2eb57
fixup 2022-10-06 19:23:31 +01:00
Starbeamrainbowlabs 9f3ae96894
finish wiring for --water-size 2022-10-06 19:21:50 +01:00
Starbeamrainbowlabs 5dac70aa08
typo 2022-10-06 19:17:03 +01:00
Starbeamrainbowlabs 2960d3b645
exception → warning 2022-10-06 18:26:40 +01:00
Starbeamrainbowlabs 0ee6703c1e
Add todo and comment 2022-10-03 19:06:56 +01:00
Starbeamrainbowlabs 2b182214ea
typo 2022-10-03 17:53:10 +01:00
Starbeamrainbowlabs 92c380bff5
fiddle with Conv2DTranspose 
you need to set the `stride` argument to actually get it to upscale..... :P
 2022-10-03 17:51:41 +01:00
Starbeamrainbowlabs d544553800
fixup 2022-10-03 17:33:06 +01:00
Starbeamrainbowlabs 058e3b6248
model_segmentation: cast float → int 2022-10-03 17:31:36 +01:00
Starbeamrainbowlabs 04e5ae0c45
model_segmentation: redo reshape 
much cheese was applied :P
 2022-10-03 17:27:52 +01:00
Starbeamrainbowlabs deffe69202
typo 2022-10-03 16:59:36 +01:00
Starbeamrainbowlabs fc6d2dabc9
Upscale first, THEN convnext... 2022-10-03 16:38:43 +01:00
Starbeamrainbowlabs 6a0790ff50
convnext_inverse: add returns; change ordering 2022-10-03 16:32:09 +01:00
Starbeamrainbowlabs e51087d0a9
add reshape layer 2022-09-28 18:22:48 +01:00
Starbeamrainbowlabs a336cdee90
and continues 2022-09-28 18:18:10 +01:00
Starbeamrainbowlabs de47a883d9
missing units 2022-09-28 18:17:22 +01:00
Starbeamrainbowlabs b5e08f92fe
the long night continues 2022-09-28 18:14:09 +01:00
Starbeamrainbowlabs dc159ecfdb
and again 2022-09-28 18:11:46 +01:00
Starbeamrainbowlabs 4cf0485e32
fixup... again 2022-09-28 18:10:11 +01:00
Starbeamrainbowlabs 030d8710b6
fixup 2022-09-28 18:08:31 +01:00
Starbeamrainbowlabs 4ee7f2a0d6
add water thresholding 2022-09-28 18:07:26 +01:00
Starbeamrainbowlabs 404dc30f08
and again 2022-09-28 17:39:09 +01:00
Starbeamrainbowlabs 4cd8fc6ded
segmentation: param name fix 2022-09-28 17:37:42 +01:00
Starbeamrainbowlabs 41ba980d69
segmentationP implement dataset parser 2022-09-28 17:19:21 +01:00
Starbeamrainbowlabs d618e6f8d7
pretrain-predict: params.json → metadata.jsonl 2022-09-28 16:35:22 +01:00
Starbeamrainbowlabs e9e6139c7a
typo 2022-09-28 16:28:18 +01:00
Starbeamrainbowlabs 3dee3d8908
update cli help 2022-09-28 16:23:47 +01:00
Starbeamrainbowlabs d765b3b14e
fix crash 2022-09-27 18:43:43 +01:00
Starbeamrainbowlabs f4d1d1d77e
just wh 2022-09-27 18:25:45 +01:00
Starbeamrainbowlabs 4c24d69ae6
$d → +d 2022-09-27 18:17:07 +01:00
Starbeamrainbowlabs cdb19b4d9f
fixup 2022-09-27 18:13:21 +01:00
Starbeamrainbowlabs c4d3c16873
add some logging 2022-09-27 18:10:58 +01:00
Starbeamrainbowlabs 3772c3227e
fixup 2022-09-27 17:57:21 +01:00
Starbeamrainbowlabs dbfa45a016
write params.json properly 2022-09-27 17:49:54 +01:00
Starbeamrainbowlabs a5455dc22a
fix import 2022-09-27 17:41:24 +01:00
Starbeamrainbowlabs d6ff3fb2ce
pretrain_predict fix write mode 2022-09-27 17:38:12 +01:00