sbrl-github/research-rainfallradar
1
0
Fork 0
mirror of https://github.com/sbrl/research-rainfallradar synced 2024-06-30 18:44:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
320 commits 1 branch 0 tags 2.1 MiB
Commit graph

320 commits

This branch
This branch
All branches
Author SHA1 Message Date
Starbeamrainbowlabs ce194d9227
slurm: customise log file names 2022-11-10 21:09:34 +00:00
Starbeamrainbowlabs 9384b89165
model_segmentation: spare → normal crossentropy, activation functions at end 2022-11-10 20:53:37 +00:00
Starbeamrainbowlabs b6676e7361
switch from sparse to normal crossentropy 2022-11-10 20:50:56 +00:00
Starbeamrainbowlabs d8be26d476
Merge branch 'main' of git.starbeamrainbowlabs.com:sbrl/PhD-Rainfall-Radar 2022-11-10 20:49:01 +00:00
Starbeamrainbowlabs b03388de60
dataset_segmenter: DEBUG: fix water shape 2022-11-10 20:48:21 +00:00
Starbeamrainbowlabs daf691bf43
typo 2022-11-10 19:55:00 +00:00
Starbeamrainbowlabs 0aa2ce19f5
read_metadata: support file inputs as well as dirs 2022-11-10 19:53:30 +00:00
Starbeamrainbowlabs aa7d9b8cf6
fixup 2022-11-10 19:46:09 +00:00
Starbeamrainbowlabs 0894bd09e8
train_predict: add error message for parrams.json not found 2022-11-10 19:45:41 +00:00
Starbeamrainbowlabs e519b0adb3
GzipChildProcess: spawn-stream is buggy IIRC 2022-11-09 16:43:05 +00:00
Starbeamrainbowlabs ddbf2cb734
slurm-process: -n28 → --exclusive 2022-11-04 17:42:50 +00:00
Starbeamrainbowlabs a762664063
slurm-process: -n28 fo uniq call 2022-11-04 17:23:20 +00:00
Starbeamrainbowlabs 0166b4d09e
slurm-process: change log file names 2022-11-04 17:11:10 +00:00
Starbeamrainbowlabs 0353072d15
allow pretrain to run on gpu 
we've slashed the size of the 2nd encoder, so ti should fit naow?
 2022-11-04 17:02:07 +00:00
Starbeamrainbowlabs 44ad51f483
CallbackNBatchCsv: bugfix .sort() → sorted() 2022-11-04 16:40:21 +00:00
Starbeamrainbowlabs 4dddcfcb42
pretrain_predict: missing \n 2022-11-04 16:01:28 +00:00
Starbeamrainbowlabs 1375201c5f
CallbackNBatchCsv: open_handle mode 2022-11-03 18:29:00 +00:00
Starbeamrainbowlabs 3206d6b7e7
slurm: rename segmenter job name 2022-11-03 17:12:27 +00:00
Starbeamrainbowlabs f2ae74ce7b
how could I be so stupid..... round 2 2022-11-02 17:38:26 +00:00
Starbeamrainbowlabs 441ad92b12
slurm: fixup 2022-11-01 19:57:15 +00:00
Starbeamrainbowlabs bc0e5f05a8
slurm: fixup 2022-11-01 19:55:04 +00:00
Starbeamrainbowlabs 784b8ed35c
recordify: catch NaN --count-file 2022-11-01 19:53:21 +00:00
Starbeamrainbowlabs c17a4ca05a
slurm: fix sanity logic 2022-11-01 19:38:04 +00:00
Starbeamrainbowlabs 79b231198f
slurm-process: check input files are readable 2022-11-01 19:03:37 +00:00
Starbeamrainbowlabs a69fa9f0f3
slurm: rename 2022-11-01 18:59:55 +00:00
Starbeamrainbowlabs f8341e7d89
slurm: add .log 2022-11-01 18:59:15 +00:00
Starbeamrainbowlabs fecc63b6a2
wrangler: write high-level job file 2022-11-01 18:56:27 +00:00
Starbeamrainbowlabs 91152ebb1c
wrangler:recordify update cli help 
we only output .jsonl.gz to a DIRECTORY, so update cli help to reflect this
 2022-11-01 18:29:47 +00:00
Starbeamrainbowlabs 5f8d6dc6ea
Add metrics every 64 batches 
this is important, because with large batches it can be difficult to tell what's happening inside each epoch.
 2022-10-31 19:26:10 +00:00
Starbeamrainbowlabs cf872ef739
how could I be so *stupid*...... 2022-10-31 18:40:58 +00:00
Starbeamrainbowlabs da32d75778
make_callbacks: display steps, not samples 2022-10-31 18:36:28 +00:00
Starbeamrainbowlabs dfef7db421
moar debugging 2022-10-31 18:26:34 +00:00
Starbeamrainbowlabs 172cf9d8ce
tweak 2022-10-31 18:19:43 +00:00
Starbeamrainbowlabs dbe35ee943
loss: comment l2 norm 2022-10-31 18:09:03 +00:00
Starbeamrainbowlabs 5e60319024
fixup 2022-10-31 17:56:49 +00:00
Starbeamrainbowlabs b986b069e2
debug party time 2022-10-31 17:50:29 +00:00
Starbeamrainbowlabs 458faa96d2
loss: fixup 2022-10-31 17:18:21 +00:00
Starbeamrainbowlabs 55dc05e8ce
contrastive: comment weights that aren't needed 2022-10-31 16:26:48 +00:00
Starbeamrainbowlabs 33391eaf16
train_predict/jsonl: don't argmax 
I'm interested inthe raw values
 2022-10-26 17:21:19 +01:00
Starbeamrainbowlabs 74f2cdb900
train_predict: .list() → .tolist() 2022-10-26 17:12:36 +01:00
Starbeamrainbowlabs 4f9d543695
train_predict: don't pass model_code 
it's redundant
 2022-10-26 17:11:36 +01:00
Starbeamrainbowlabs 1b489518d0
segmenter: add LayerStack2Image to custom_objects 2022-10-26 17:05:50 +01:00
Starbeamrainbowlabs 48ae8a5c20
LossContrastive: normalise features as per the paper 2022-10-26 16:52:56 +01:00
Starbeamrainbowlabs 843cc8dc7b
contrastive: rewrite the loss function. 
The CLIP paper *does* kinda make sense I think
 2022-10-26 16:45:45 +01:00
Starbeamrainbowlabs fad1399c2d
convnext: whitespace 2022-10-26 16:45:20 +01:00
Starbeamrainbowlabs 1d872cb962
contrastive: fix initial temperature value 
It should be 1/0.07, but we had it set to 0.07......
 2022-10-26 16:45:01 +01:00
Starbeamrainbowlabs f994d449f1
Layer2Image: fix 2022-10-25 21:32:17 +01:00
Starbeamrainbowlabs 6a29105f56
model_segmentation: stack not reshape 2022-10-25 21:25:15 +01:00
Starbeamrainbowlabs 98417a3e06
prepare for NCE loss 
.....but Tensorflow's implementation looks to be for supervised models :-(
 2022-10-25 21:15:05 +01:00
Starbeamrainbowlabs bb0679a509
model_segmentation: don't softmax twice 2022-10-25 21:11:48 +01:00