290 commits

Author	SHA1	Message	Date
Starbeamrainbowlabs	5f8d6dc6ea	Add metrics every 64 batches ... this is important, because with large batches it can be difficult to tell what's happening inside each epoch.	2022-10-31 19:26:10 +00:00
Starbeamrainbowlabs	cf872ef739	how could I be so *stupid*......	2022-10-31 18:40:58 +00:00
Starbeamrainbowlabs	da32d75778	make_callbacks: display steps, not samples	2022-10-31 18:36:28 +00:00
Starbeamrainbowlabs	dfef7db421	moar debugging	2022-10-31 18:26:34 +00:00
Starbeamrainbowlabs	172cf9d8ce	tweak	2022-10-31 18:19:43 +00:00
Starbeamrainbowlabs	dbe35ee943	loss: comment l2 norm	2022-10-31 18:09:03 +00:00
Starbeamrainbowlabs	5e60319024	fixup	2022-10-31 17:56:49 +00:00
Starbeamrainbowlabs	b986b069e2	debug party time	2022-10-31 17:50:29 +00:00
Starbeamrainbowlabs	458faa96d2	loss: fixup	2022-10-31 17:18:21 +00:00
Starbeamrainbowlabs	55dc05e8ce	contrastive: comment weights that aren't needed	2022-10-31 16:26:48 +00:00
Starbeamrainbowlabs	33391eaf16	train_predict/jsonl: don't argmax ... I'm interested inthe raw values	2022-10-26 17:21:19 +01:00
Starbeamrainbowlabs	74f2cdb900	train_predict: .list() → .tolist()	2022-10-26 17:12:36 +01:00
Starbeamrainbowlabs	4f9d543695	train_predict: don't pass model_code ... it's redundant	2022-10-26 17:11:36 +01:00
Starbeamrainbowlabs	1b489518d0	segmenter: add LayerStack2Image to custom_objects	2022-10-26 17:05:50 +01:00
Starbeamrainbowlabs	48ae8a5c20	LossContrastive: normalise features as per the paper	2022-10-26 16:52:56 +01:00
Starbeamrainbowlabs	843cc8dc7b	contrastive: rewrite the loss function. ... The CLIP paper *does* kinda make sense I think	2022-10-26 16:45:45 +01:00
Starbeamrainbowlabs	fad1399c2d	convnext: whitespace	2022-10-26 16:45:20 +01:00
Starbeamrainbowlabs	1d872cb962	contrastive: fix initial temperature value ... It should be 1/0.07, but we had it set to 0.07......	2022-10-26 16:45:01 +01:00
Starbeamrainbowlabs	f994d449f1	Layer2Image: fix	2022-10-25 21:32:17 +01:00
Starbeamrainbowlabs	6a29105f56	model_segmentation: stack not reshape	2022-10-25 21:25:15 +01:00
Starbeamrainbowlabs	98417a3e06	prepare for NCE loss ... .....but Tensorflow's implementation looks to be for supervised models :-(	2022-10-25 21:15:05 +01:00
Starbeamrainbowlabs	bb0679a509	model_segmentation: don't softmax twice	2022-10-25 21:11:48 +01:00
Starbeamrainbowlabs	f2e2ca1484	model_contrastive: make water encoder significantly shallower	2022-10-24 20:52:31 +01:00
Starbeamrainbowlabs	a6b07a49cb	count water/nowater pixels in Jupyter Notebook	2022-10-24 18:05:34 +01:00
Starbeamrainbowlabs	a8b101bdae	dataset_predict: add shape_water_desired	2022-10-24 18:05:13 +01:00
Starbeamrainbowlabs	587c1dfafa	train_predict: revamp jsonl handling	2022-10-21 16:53:08 +01:00
Starbeamrainbowlabs	8195318a42	SparseCategoricalAccuracy: losses → metrics	2022-10-21 16:51:20 +01:00
Starbeamrainbowlabs	612735aaae	rename shuffle arg	2022-10-21 16:35:45 +01:00
Starbeamrainbowlabs	c98d8d05dd	segmentation: use the right accuracy	2022-10-21 16:17:05 +01:00
Starbeamrainbowlabs	bb0258f5cd	flip squeeze operator ordering	2022-10-21 15:38:57 +01:00
Starbeamrainbowlabs	af26964c6a	batched_iterator: reset i_item after every time	2022-10-21 15:35:43 +01:00
Starbeamrainbowlabs	c5b1501dba	train-predict fixup	2022-10-21 15:27:39 +01:00
Starbeamrainbowlabs	42aea7a0cc	plt.close() fixup	2022-10-21 15:23:54 +01:00
Starbeamrainbowlabs	12dad3bc87	vis/segmentation: fix titles	2022-10-21 15:22:35 +01:00
Starbeamrainbowlabs	0cb2de5d06	train-preedict: close matplotlib after we've finished ... they act like file handles	2022-10-21 15:19:31 +01:00
Starbeamrainbowlabs	81e53efd9c	PNG: create output dir if doesn't exist	2022-10-21 15:17:39 +01:00
Starbeamrainbowlabs	3f7db6fa78	fix embedding confusion	2022-10-21 15:15:59 +01:00
Starbeamrainbowlabs	847cd97ec4	fixup	2022-10-21 14:26:58 +01:00
Starbeamrainbowlabs	0e814b7e98	Contraster → Segmenter	2022-10-21 14:25:43 +01:00
Starbeamrainbowlabs	1b658a1b7c	train-predict: can't destructure array when iterating generator ... ....it seems to lead to undefined behaviour or something	2022-10-20 19:34:04 +01:00
Starbeamrainbowlabs	aed2348a95	train_predict: fixup	2022-10-20 15:42:33 +01:00
Starbeamrainbowlabs	cc6679c609	batch data; use generator	2022-10-20 15:22:29 +01:00
Starbeamrainbowlabs	d306853c42	use right daataset	2022-10-20 15:16:24 +01:00
Starbeamrainbowlabs	59cfa4a89a	basename paths	2022-10-20 15:11:14 +01:00
Starbeamrainbowlabs	4d8ae21a45	update cli help text	2022-10-19 17:31:42 +01:00
Starbeamrainbowlabs	200076596b	finish train_predict	2022-10-19 17:26:40 +01:00
Starbeamrainbowlabs	488f78fca5	pretrain_predict: default to parallel_reads=0	2022-10-19 16:59:45 +01:00
Starbeamrainbowlabs	63e909d9fc	datasets: add shuffle=True/False to get_filepaths. ... This is important because otherwise it SCAMBLES the filenames, which is a disaster for making predictions in the right order....!	2022-10-19 16:52:07 +01:00
Starbeamrainbowlabs	fe43ddfbf9	start implementing driver for train_predict, but not finished yet	2022-10-18 19:37:55 +01:00
Starbeamrainbowlabs	b3ea189d37	segmentation: softmax the output	2022-10-13 21:02:57 +01:00
Starbeamrainbowlabs	f121bfb981	fixup summaryfile	2022-10-13 17:54:42 +01:00
Starbeamrainbowlabs	5c35c0cee4	model_segmentation: document; remove unused args	2022-10-13 17:50:16 +01:00
Starbeamrainbowlabs	f12e6ab905	No need for a CLI arg for feature_dim_in - metadata should contain this	2022-10-13 17:37:16 +01:00
Starbeamrainbowlabs	e201372252	write quick Jupyter notebook to test data ... ....I'm paranoid	2022-10-13 17:27:17 +01:00
Starbeamrainbowlabs	ae53130e66	layout	2022-10-13 14:54:20 +01:00
Starbeamrainbowlabs	7933564c66	typo	2022-10-12 17:33:54 +01:00
Starbeamrainbowlabs	dbe4fb0eab	train: add slurm job file	2022-10-12 17:27:10 +01:00
Starbeamrainbowlabs	6423bf6702	LayerConvNeXtGamma: avoid adding an EagerTensor to config ... Very weird how this is a problem when it wasn't before..	2022-10-12 17:12:07 +01:00
Starbeamrainbowlabs	32f5200d3b	pass model_arch properly	2022-10-12 16:50:06 +01:00
Starbeamrainbowlabs	5933fb1061	fixup	2022-10-11 19:23:41 +01:00
Starbeamrainbowlabs	c45b90764e	segmentation: adds xxtiny, but unsure if it's small enough	2022-10-11 19:22:37 +01:00
Starbeamrainbowlabs	f4a2c742d9	typo	2022-10-11 19:19:23 +01:00
Starbeamrainbowlabs	11f91a7cf4	train: add --arch; default to convnext_i_xtiny	2022-10-11 19:18:01 +01:00
Starbeamrainbowlabs	5666c5a0d9	typo	2022-10-10 18:12:51 +01:00
Starbeamrainbowlabs	131c0a0a5b	pretrain-predict: create dir if not exists	2022-10-10 18:00:55 +01:00
Starbeamrainbowlabs	deede32241	slurm-pretrain: limit memory usage	2022-10-10 17:45:29 +01:00
Starbeamrainbowlabs	13a8f3f511	pretrain-predict: only queue pretrain-plot if I we output jsonl	2022-10-10 17:11:10 +01:00
Starbeamrainbowlabs	ffcb2e3735	pretrain-predict: queue for the actual input	2022-10-10 16:53:28 +01:00
Starbeamrainbowlabs	f883986eaa	Bugfix: modeset to enable TFRecordWriter instead of bare handle	2022-10-06 20:07:59 +01:00
Starbeamrainbowlabs	e9a8e2eb57	fixup	2022-10-06 19:23:31 +01:00
Starbeamrainbowlabs	9f3ae96894	finish wiring for --water-size	2022-10-06 19:21:50 +01:00
Starbeamrainbowlabs	5dac70aa08	typo	2022-10-06 19:17:03 +01:00
Starbeamrainbowlabs	2960d3b645	exception → warning	2022-10-06 18:26:40 +01:00
Starbeamrainbowlabs	0ee6703c1e	Add todo and comment	2022-10-03 19:06:56 +01:00
Starbeamrainbowlabs	2b182214ea	typo	2022-10-03 17:53:10 +01:00
Starbeamrainbowlabs	92c380bff5	fiddle with Conv2DTranspose ... you need to set the `stride` argument to actually get it to upscale..... :P	2022-10-03 17:51:41 +01:00
Starbeamrainbowlabs	d544553800	fixup	2022-10-03 17:33:06 +01:00
Starbeamrainbowlabs	058e3b6248	model_segmentation: cast float → int	2022-10-03 17:31:36 +01:00
Starbeamrainbowlabs	04e5ae0c45	model_segmentation: redo reshape ... much cheese was applied :P	2022-10-03 17:27:52 +01:00
Starbeamrainbowlabs	deffe69202	typo	2022-10-03 16:59:36 +01:00
Starbeamrainbowlabs	fc6d2dabc9	Upscale first, THEN convnext...	2022-10-03 16:38:43 +01:00
Starbeamrainbowlabs	6a0790ff50	convnext_inverse: add returns; change ordering	2022-10-03 16:32:09 +01:00
Starbeamrainbowlabs	fe813cb46d	slurm predict: fix plotting subcall	2022-10-03 16:03:26 +01:00
Starbeamrainbowlabs	e51087d0a9	add reshape layer	2022-09-28 18:22:48 +01:00
Starbeamrainbowlabs	a336cdee90	and continues	2022-09-28 18:18:10 +01:00
Starbeamrainbowlabs	de47a883d9	missing units	2022-09-28 18:17:22 +01:00
Starbeamrainbowlabs	b5e08f92fe	the long night continues	2022-09-28 18:14:09 +01:00
Starbeamrainbowlabs	dc159ecfdb	and again	2022-09-28 18:11:46 +01:00
Starbeamrainbowlabs	4cf0485e32	fixup... again	2022-09-28 18:10:11 +01:00
Starbeamrainbowlabs	030d8710b6	fixup	2022-09-28 18:08:31 +01:00
Starbeamrainbowlabs	4ee7f2a0d6	add water thresholding	2022-09-28 18:07:26 +01:00
Starbeamrainbowlabs	404dc30f08	and again	2022-09-28 17:39:09 +01:00
Starbeamrainbowlabs	4cd8fc6ded	segmentation: param name fix	2022-09-28 17:37:42 +01:00
Starbeamrainbowlabs	41ba980d69	segmentationP implement dataset parser	2022-09-28 17:19:21 +01:00
Starbeamrainbowlabs	d618e6f8d7	pretrain-predict: params.json → metadata.jsonl	2022-09-28 16:35:22 +01:00
Starbeamrainbowlabs	e9e6139c7a	typo	2022-09-28 16:28:18 +01:00
Starbeamrainbowlabs	3dee3d8908	update cli help	2022-09-28 16:23:47 +01:00
Starbeamrainbowlabs	b836f7f70c	again	2022-09-27 19:06:41 +01:00
Starbeamrainbowlabs	52dff130dd	slurm pretraian predict: moar logging	2022-09-27 19:05:54 +01:00
Starbeamrainbowlabs	2d24174e0a	slurm pretrain predict: add $OUTPUT	2022-09-27 18:58:02 +01:00