486 commits

Author	SHA1	Message	Date
Starbeamrainbowlabs	4b7df39fac	bugfix	2023-01-09 18:25:16 +00:00
Starbeamrainbowlabs	581006cbe6	dlr: save checkpoints	2023-01-09 18:03:23 +00:00
Starbeamrainbowlabs	52cf66ca32	start working on a quick encoder test, but it's far from finished	2023-01-06 19:55:52 +00:00
Starbeamrainbowlabs	f080af0b57	identity test: plot binarised heightmap	2023-01-06 19:03:06 +00:00
Starbeamrainbowlabs	36859746ff	dlr: fix crash	2023-01-06 17:13:35 +00:00
Starbeamrainbowlabs	bcf198d47b	mono identity test: output	2023-01-06 17:08:18 +00:00
Starbeamrainbowlabs	db0b010814	slur dlr: log file names correct	2023-01-05 19:47:51 +00:00
Starbeamrainbowlabs	e01ecfb615	slurm dlr: fix output dir	2023-01-05 19:42:42 +00:00
Starbeamrainbowlabs	aa76d754c1	slurm dlr: fix pathing	2023-01-05 19:35:56 +00:00
Starbeamrainbowlabs	67b8a2c6c0	spaces → spaces	2023-01-05 19:17:44 +00:00
Starbeamrainbowlabs	56a501f8a9	weights="imagenet" only works with 3 image channels	2023-01-05 19:09:31 +00:00
Starbeamrainbowlabs	4563fe6b27	dpl: fix moar crashes	2023-01-05 19:03:44 +00:00
Starbeamrainbowlabs	fefeb5d531	fix water depth fiddling	2023-01-05 19:01:20 +00:00
Starbeamrainbowlabs	46d1f5e4e0	dataset_mono: fix dataset parsing	2023-01-05 19:00:52 +00:00
Starbeamrainbowlabs	aca7b83a78	dataset_mono: fix sizing ... it didn't account for rainfall_scale_up	2023-01-05 18:38:47 +00:00
Starbeamrainbowlabs	19bb2fcac0	debug	2023-01-05 18:32:22 +00:00
Starbeamrainbowlabs	6a4f68a055	missing import	2023-01-05 18:26:33 +00:00
Starbeamrainbowlabs	0d4cc63b76	dl rainfall: fix env var name	2023-01-05 17:42:20 +00:00
Starbeamrainbowlabs	dd79fb6e68	fixup	2023-01-05 17:09:09 +00:00
Starbeamrainbowlabs	11ccd4cbee	slurm deeplab rainfall: fix variable naming	2023-01-05 17:08:57 +00:00
Starbeamrainbowlabs	c17e53ca75	deeplabv3+ for rainfall	2022-12-16 19:52:59 +00:00
Starbeamrainbowlabs	677e39f820	work on slurm for deeplabv3+ rainfall, but it's NOT FINISHED YET	2022-12-16 19:52:44 +00:00
Starbeamrainbowlabs	423e277ed1	add comment	2022-12-15 19:33:25 +00:00
Starbeamrainbowlabs	ef5071b569	DeepLabV3+: start working on version for rainfall radar, but it's not finished yet	2022-12-15 19:33:14 +00:00
Starbeamrainbowlabs	15a3150127	DeepLabV3+: close each matplotlib figure after writing it	2022-12-15 19:14:07 +00:00
Starbeamrainbowlabs	6ce121f861	DeepLabV3+: have argument for number of channels	2022-12-14 17:36:30 +00:00
Starbeamrainbowlabs	1dc2ec3a46	DeepLabV3+: pathing.... again	2022-12-13 18:51:09 +00:00
Starbeamrainbowlabs	eb47f8f544	dataset_mono: adjust to suit DeepLabV3+ too	2022-12-13 18:37:38 +00:00
Starbeamrainbowlabs	440e693dfc	DeepLabv3+: fix pathing again	2022-12-13 18:26:00 +00:00
Starbeamrainbowlabs	7e1f271bf4	deeplabv3+: fix colourmap	2022-12-13 14:02:10 +00:00
Starbeamrainbowlabs	4d8ce792c9	ddeeplabv3+: fix imports/pathing errors	2022-12-13 13:38:27 +00:00
Starbeamrainbowlabs	fc43f145c2	if not	2022-12-13 13:28:09 +00:00
Starbeamrainbowlabs	d907dc48e5	DeepLabv3+: add logging	2022-12-13 13:20:16 +00:00
Starbeamrainbowlabs	be4d928319	deeplabv3+: chmod +x	2022-12-13 13:06:42 +00:00
Starbeamrainbowlabs	91846079b2	deeplabv3+ tesst: add shebang	2022-12-13 12:56:14 +00:00
Starbeamrainbowlabs	96e260fe82	slurm: add job file for deeplabv3 test	2022-12-12 19:31:49 +00:00
Starbeamrainbowlabs	8866960017	TEST SCRIPT: deeplabv3 ... ref https://keras.io/examples/vision/deeplabv3_plus/ dataset ref https://drive.google.com/uc?id=1B9A9UCJYMwTL4oBEo4RZfbMZMaZhKJaz (the code is *terrible* spaghetti....!)	2022-12-12 19:20:07 +00:00
Starbeamrainbowlabs	4e4d42a281	LossDice: add comment	2022-12-12 18:34:20 +00:00
Starbeamrainbowlabs	449bc425a7	LossDice: explicitly cast inputs to float32	2022-12-12 17:20:32 +00:00
Starbeamrainbowlabs	dbf8f5617c	drop activation function in last layers	2022-12-12 17:20:04 +00:00
Starbeamrainbowlabs	bcd2f1251e	LossDice: Do 1 - thing instead of -thing	2022-12-09 19:41:32 +00:00
Starbeamrainbowlabs	d0dbc50bb7	debug	2022-12-09 19:33:28 +00:00
Starbeamrainbowlabs	2142bb039c	again	2022-12-09 19:30:01 +00:00
Starbeamrainbowlabs	7000b0f193	fixup	2022-12-09 19:23:35 +00:00
Starbeamrainbowlabs	85012d0616	fixup	2022-12-09 19:18:03 +00:00
Starbeamrainbowlabs	719d8e9819	strip channels layer at end	2022-12-09 19:11:00 +00:00
Starbeamrainbowlabs	0129c35a35	LossDice: remove weird K.* functions	2022-12-09 19:06:26 +00:00
Starbeamrainbowlabs	659fc97fd4	fix crash	2022-12-09 18:39:27 +00:00
Starbeamrainbowlabs	e22c0981e6	actually use dice loss	2022-12-09 18:35:17 +00:00
Starbeamrainbowlabs	649c262960	mono: switch loss from crossentropy to dice	2022-12-09 18:13:37 +00:00
Starbeamrainbowlabs	7fd7c750d6	jupyter: identity test ... status: FAILED, as usual....! Don't worry though, 'cause we has a *planses*..... MUHAHAHAHAHAHAHA * cue evil laugh *	2022-12-09 18:07:56 +00:00
Starbeamrainbowlabs	cf9e8aa237	jupyter: convnext-mono identity test	2022-12-09 15:50:27 +00:00
Starbeamrainbowlabs	2a1772a211	confvnext_intrevse: add shallow	2022-12-08 19:10:12 +00:00
Starbeamrainbowlabs	c27869630a	I hate VSCode's git commit interface ... it doesn't let you ammend	2022-12-08 18:58:54 +00:00
Starbeamrainbowlabs	b3345963f3	missing arg pass	2022-12-08 18:58:32 +00:00
Starbeamrainbowlabs	3dde9b69da	fixup	2022-12-08 18:56:32 +00:00
Starbeamrainbowlabs	6fce39f696	WHY?!?!?!	2022-12-08 18:55:53 +00:00
Starbeamrainbowlabs	26766366fc	I hate the python code intelligence ... it's bad	2022-12-08 18:55:15 +00:00
Starbeamrainbowlabs	ff56f591c7	I hate python	2022-12-08 18:53:37 +00:00
Starbeamrainbowlabs	d37e7224f5	train-mono: tidy up arg passing	2022-12-08 18:47:03 +00:00
Starbeamrainbowlabs	b53db648bf	fixup	2022-12-08 18:31:42 +00:00
Starbeamrainbowlabs	18c0210704	typo	2022-12-08 17:00:25 +00:00
Starbeamrainbowlabs	a3c9416cf0	LossCrossentropy: don't sum	2022-12-08 16:57:11 +00:00
Starbeamrainbowlabs	08046340f4	dataset_mono: normalise heightmap	2022-12-08 16:10:34 +00:00
Starbeamrainbowlabs	d997157f55	dataset_mono: log when using heightmap	2022-12-06 19:30:11 +00:00
Starbeamrainbowlabs	468c150570	slurm-train-mono: add HEIGHTMAP	2022-12-06 19:28:06 +00:00
Starbeamrainbowlabs	d0f2e3d730	readfile: do transparent gzip by default ... ....but there's a glad to turn it off if needed	2022-12-06 19:27:39 +00:00
Starbeamrainbowlabs	eac6472c97	Implement support for (optionally) taking a heightmap in	2022-12-06 18:55:58 +00:00
Starbeamrainbowlabs	f92b2b3472	according to the equation it looks like it's 2	2022-12-02 17:22:46 +00:00
Starbeamrainbowlabs	cad82cd1bc	CBAM: unsure if it's 1 ro 3 dense ayers in the shared mlp	2022-12-02 17:21:13 +00:00
Starbeamrainbowlabs	62f6a993bb	implement CBAM, but it's UNTESTED ... Convolutional Block Attention Module.	2022-12-02 17:17:45 +00:00
Starbeamrainbowlabs	9d666c3b38	train mono: type=int → float	2022-12-01 15:39:44 +00:00
Starbeamrainbowlabs	53dfa32685	model_mono: log learning rate	2022-12-01 15:10:51 +00:00
Starbeamrainbowlabs	c384d55dff	add arg to adjust learning rate	2022-11-29 20:55:00 +00:00
Starbeamrainbowlabs	8e23e9d341	model_segmenter: we're no longer using sparse	2022-11-29 19:28:27 +00:00
Starbeamrainbowlabs	9a2b4c6838	dsseg: fix reshape/onehot ordering	2022-11-29 19:28:13 +00:00
Starbeamrainbowlabs	df774146d9	dataset_segmenter: reshape, not squeeze	2022-11-29 19:24:54 +00:00
Starbeamrainbowlabs	77b8a1a8db	dataset_segmenter: squeeze	2022-11-29 19:16:15 +00:00
Starbeamrainbowlabs	2258b5a229	slurm-train: reduce RAM required by 10GB	2022-11-29 19:15:34 +00:00
Starbeamrainbowlabs	01101ad30b	losscrossentropy: return the reduced value * facepalm *	2022-11-29 19:07:08 +00:00
Starbeamrainbowlabs	ff65393e78	log file naming update	2022-11-29 18:41:14 +00:00
Starbeamrainbowlabs	37f196a785	LossCrossentropy: add kwargs	2022-11-29 15:40:35 +00:00
Starbeamrainbowlabs	838ff56a3b	mono: fix loading checkpoint	2022-11-29 15:25:11 +00:00
Starbeamrainbowlabs	dba6cbffcd	WHY. * facepalms *	2022-11-28 19:33:42 +00:00
Starbeamrainbowlabs	57b8eb93fb	fixup	2022-11-28 19:09:35 +00:00
Starbeamrainbowlabs	6640a41bb7	almost got it....? it's not what I expected....!	2022-11-28 19:08:50 +00:00
Starbeamrainbowlabs	f48473b703	fixup	2022-11-28 19:00:11 +00:00
Starbeamrainbowlabs	f6feb125e3	this iss ome serious debugging. ... This commit will produce an extremely large volume of output.	2022-11-28 18:57:41 +00:00
Starbeamrainbowlabs	09f81b0746	train_mono: debug ... this commit will generate a large amount of debug output.	2022-11-28 16:46:17 +00:00
Starbeamrainbowlabs	f39e4ade70	LayerConvNextGamma: fix config serialisation bug ... .....this is unlikely to be the problem as this bug is in an unused code path.	2022-11-25 21:16:31 +00:00
Starbeamrainbowlabs	e7410fb480	train_mono_predict: limit label size to 64x64 ... that's the size the model predicts	2022-11-25 17:47:17 +00:00
Starbeamrainbowlabs	51dd484d13	fixup	2022-11-25 16:55:45 +00:00
Starbeamrainbowlabs	884c4eb150	rainfall_stats: formatting again	2022-11-24 19:08:07 +00:00
Starbeamrainbowlabs	bfe038086c	rainfall_stats: formatting	2022-11-24 19:07:44 +00:00
Starbeamrainbowlabs	7dba03200f	fixup	2022-11-24 19:06:48 +00:00
Starbeamrainbowlabs	e5258b9c66	typo	2022-11-24 19:06:13 +00:00
Starbeamrainbowlabs	64d646bb13	rainfall_stats: formatting	2022-11-24 19:05:35 +00:00
Starbeamrainbowlabs	675c7a7448	fixup	2022-11-24 19:03:28 +00:00
Starbeamrainbowlabs	afc1cdcf02	fixup	2022-11-24 19:02:58 +00:00
Starbeamrainbowlabs	e4bea89c89	typo	2022-11-24 19:01:52 +00:00
Starbeamrainbowlabs	a40cbe8705	rainfall_stats: remove unused imports	2022-11-24 19:01:18 +00:00
Starbeamrainbowlabs	fe57d6aab2	rainfall_stats: initial implementation ... this might reveal why we are having problems. If most/all the rainfall radar data is v small numbers, normalising might help.	2022-11-24 18:58:16 +00:00
Starbeamrainbowlabs	3131b4f7b3	debug2	2022-11-24 18:25:32 +00:00
Starbeamrainbowlabs	d55a13f536	debug	2022-11-24 18:24:03 +00:00
Starbeamrainbowlabs	1f60f2a580	do_argmax	2022-11-24 18:11:03 +00:00
Starbeamrainbowlabs	6c09d5254d	fixup	2022-11-24 17:57:48 +00:00
Starbeamrainbowlabs	54a841efe9	train_mono_predict: convert to correct format	2022-11-24 17:56:07 +00:00
Starbeamrainbowlabs	105dc5bc56	missing kwargs	2022-11-24 17:51:29 +00:00
Starbeamrainbowlabs	1e1d6dd273	fixup	2022-11-24 17:48:19 +00:00
Starbeamrainbowlabs	011e0aef78	update cli docs	2022-11-24 16:38:07 +00:00
Starbeamrainbowlabs	773944f9fa	train_mono_predict: initial implementation	2022-11-24 16:33:50 +00:00
Starbeamrainbowlabs	d31326cb30	slurm train mono: fix partition name	2022-11-22 17:02:02 +00:00
Starbeamrainbowlabs	ce28ac4013	slurm: add job for train_mono	2022-11-22 16:58:46 +00:00
Starbeamrainbowlabs	3a0356929c	mono: drop the sparse	2022-11-22 16:20:56 +00:00
Starbeamrainbowlabs	7e8f63f8ba	fixup	2022-11-21 19:38:24 +00:00
Starbeamrainbowlabs	ace4c8b246	dataset_mono: debug	2022-11-21 18:46:21 +00:00
Starbeamrainbowlabs	527b34942d	convnext_inverse: kernel_size 4→2	2022-11-11 19:29:37 +00:00
Starbeamrainbowlabs	0662d0854b	model_mono: fix bottleneck	2022-11-11 19:11:40 +00:00
Starbeamrainbowlabs	73acda6d9a	fix debug logging	2022-11-11 19:08:38 +00:00
Starbeamrainbowlabs	9da059d738	model shape logging	2022-11-11 19:03:37 +00:00
Starbeamrainbowlabs	00917b2698	dataset_mono: log shapes	2022-11-11 19:02:43 +00:00
Starbeamrainbowlabs	54ae88b1b4	in this entire blasted project I have yet to get the rotation of anything correct....!	2022-11-11 18:58:45 +00:00
Starbeamrainbowlabs	a7a475dcd1	debug 2	2022-11-11 18:38:07 +00:00
Starbeamrainbowlabs	bf2f6e9b64	debug logging ... it begins again	2022-11-11 18:31:40 +00:00
Starbeamrainbowlabs	481eeb3759	mono: fix dataset preprocessing ... rogue dimension	2022-11-11 18:31:27 +00:00
Starbeamrainbowlabs	9035450213	mono: instantiate right model	2022-11-11 18:28:29 +00:00
Starbeamrainbowlabs	69a2d0cf04	fixup	2022-11-11 18:27:01 +00:00
Starbeamrainbowlabs	65e801cf28	train_mono: fix crash	2022-11-11 18:26:25 +00:00
Starbeamrainbowlabs	8ac5159adc	dataset_mono: simplify param passing, onehot+threshold water depth data	2022-11-11 18:23:50 +00:00
Starbeamrainbowlabs	3a3f7e85da	typo	2022-11-11 18:03:09 +00:00
Starbeamrainbowlabs	3313f77c88	Add (untested) mono rainfall → water depth model ... * sighs * Unfortunately I can't seem to get contrastive learning to work.....	2022-11-10 22:36:11 +00:00
Starbeamrainbowlabs	ce194d9227	slurm: customise log file names	2022-11-10 21:09:34 +00:00
Starbeamrainbowlabs	9384b89165	model_segmentation: spare → normal crossentropy, activation functions at end	2022-11-10 20:53:37 +00:00
Starbeamrainbowlabs	b6676e7361	switch from sparse to normal crossentropy	2022-11-10 20:50:56 +00:00
Starbeamrainbowlabs	d8be26d476	Merge branch 'main' of git.starbeamrainbowlabs.com:sbrl/PhD-Rainfall-Radar	2022-11-10 20:49:01 +00:00
Starbeamrainbowlabs	b03388de60	dataset_segmenter: DEBUG: fix water shape	2022-11-10 20:48:21 +00:00
Starbeamrainbowlabs	daf691bf43	typo	2022-11-10 19:55:00 +00:00
Starbeamrainbowlabs	0aa2ce19f5	read_metadata: support file inputs as well as dirs	2022-11-10 19:53:30 +00:00
Starbeamrainbowlabs	aa7d9b8cf6	fixup	2022-11-10 19:46:09 +00:00
Starbeamrainbowlabs	0894bd09e8	train_predict: add error message for parrams.json not found	2022-11-10 19:45:41 +00:00
Starbeamrainbowlabs	0353072d15	allow pretrain to run on gpu ... we've slashed the size of the 2nd encoder, so ti should fit naow?	2022-11-04 17:02:07 +00:00
Starbeamrainbowlabs	44ad51f483	CallbackNBatchCsv: bugfix .sort() → sorted()	2022-11-04 16:40:21 +00:00
Starbeamrainbowlabs	4dddcfcb42	pretrain_predict: missing \n	2022-11-04 16:01:28 +00:00
Starbeamrainbowlabs	1375201c5f	CallbackNBatchCsv: open_handle mode	2022-11-03 18:29:00 +00:00
Starbeamrainbowlabs	3206d6b7e7	slurm: rename segmenter job name	2022-11-03 17:12:27 +00:00
Starbeamrainbowlabs	f2ae74ce7b	how could I be so stupid..... round 2	2022-11-02 17:38:26 +00:00
Starbeamrainbowlabs	5f8d6dc6ea	Add metrics every 64 batches ... this is important, because with large batches it can be difficult to tell what's happening inside each epoch.	2022-10-31 19:26:10 +00:00
Starbeamrainbowlabs	cf872ef739	how could I be so *stupid*......	2022-10-31 18:40:58 +00:00
Starbeamrainbowlabs	da32d75778	make_callbacks: display steps, not samples	2022-10-31 18:36:28 +00:00
Starbeamrainbowlabs	dfef7db421	moar debugging	2022-10-31 18:26:34 +00:00