265 commits

Author	SHA1	Message	Date
Starbeamrainbowlabs	587c1dfafa	train_predict: revamp jsonl handling	2022-10-21 16:53:08 +01:00
Starbeamrainbowlabs	8195318a42	SparseCategoricalAccuracy: losses → metrics	2022-10-21 16:51:20 +01:00
Starbeamrainbowlabs	612735aaae	rename shuffle arg	2022-10-21 16:35:45 +01:00
Starbeamrainbowlabs	c98d8d05dd	segmentation: use the right accuracy	2022-10-21 16:17:05 +01:00
Starbeamrainbowlabs	bb0258f5cd	flip squeeze operator ordering	2022-10-21 15:38:57 +01:00
Starbeamrainbowlabs	af26964c6a	batched_iterator: reset i_item after every time	2022-10-21 15:35:43 +01:00
Starbeamrainbowlabs	c5b1501dba	train-predict fixup	2022-10-21 15:27:39 +01:00
Starbeamrainbowlabs	42aea7a0cc	plt.close() fixup	2022-10-21 15:23:54 +01:00
Starbeamrainbowlabs	12dad3bc87	vis/segmentation: fix titles	2022-10-21 15:22:35 +01:00
Starbeamrainbowlabs	0cb2de5d06	train-preedict: close matplotlib after we've finished ... they act like file handles	2022-10-21 15:19:31 +01:00
Starbeamrainbowlabs	81e53efd9c	PNG: create output dir if doesn't exist	2022-10-21 15:17:39 +01:00
Starbeamrainbowlabs	3f7db6fa78	fix embedding confusion	2022-10-21 15:15:59 +01:00
Starbeamrainbowlabs	847cd97ec4	fixup	2022-10-21 14:26:58 +01:00
Starbeamrainbowlabs	0e814b7e98	Contraster → Segmenter	2022-10-21 14:25:43 +01:00
Starbeamrainbowlabs	1b658a1b7c	train-predict: can't destructure array when iterating generator ... ....it seems to lead to undefined behaviour or something	2022-10-20 19:34:04 +01:00
Starbeamrainbowlabs	aed2348a95	train_predict: fixup	2022-10-20 15:42:33 +01:00
Starbeamrainbowlabs	cc6679c609	batch data; use generator	2022-10-20 15:22:29 +01:00
Starbeamrainbowlabs	d306853c42	use right daataset	2022-10-20 15:16:24 +01:00
Starbeamrainbowlabs	59cfa4a89a	basename paths	2022-10-20 15:11:14 +01:00
Starbeamrainbowlabs	4d8ae21a45	update cli help text	2022-10-19 17:31:42 +01:00
Starbeamrainbowlabs	200076596b	finish train_predict	2022-10-19 17:26:40 +01:00
Starbeamrainbowlabs	488f78fca5	pretrain_predict: default to parallel_reads=0	2022-10-19 16:59:45 +01:00
Starbeamrainbowlabs	63e909d9fc	datasets: add shuffle=True/False to get_filepaths. ... This is important because otherwise it SCAMBLES the filenames, which is a disaster for making predictions in the right order....!	2022-10-19 16:52:07 +01:00
Starbeamrainbowlabs	fe43ddfbf9	start implementing driver for train_predict, but not finished yet	2022-10-18 19:37:55 +01:00
Starbeamrainbowlabs	b3ea189d37	segmentation: softmax the output	2022-10-13 21:02:57 +01:00
Starbeamrainbowlabs	f121bfb981	fixup summaryfile	2022-10-13 17:54:42 +01:00
Starbeamrainbowlabs	5c35c0cee4	model_segmentation: document; remove unused args	2022-10-13 17:50:16 +01:00
Starbeamrainbowlabs	f12e6ab905	No need for a CLI arg for feature_dim_in - metadata should contain this	2022-10-13 17:37:16 +01:00
Starbeamrainbowlabs	e201372252	write quick Jupyter notebook to test data ... ....I'm paranoid	2022-10-13 17:27:17 +01:00
Starbeamrainbowlabs	ae53130e66	layout	2022-10-13 14:54:20 +01:00
Starbeamrainbowlabs	7933564c66	typo	2022-10-12 17:33:54 +01:00
Starbeamrainbowlabs	dbe4fb0eab	train: add slurm job file	2022-10-12 17:27:10 +01:00
Starbeamrainbowlabs	6423bf6702	LayerConvNeXtGamma: avoid adding an EagerTensor to config ... Very weird how this is a problem when it wasn't before..	2022-10-12 17:12:07 +01:00
Starbeamrainbowlabs	32f5200d3b	pass model_arch properly	2022-10-12 16:50:06 +01:00
Starbeamrainbowlabs	5933fb1061	fixup	2022-10-11 19:23:41 +01:00
Starbeamrainbowlabs	c45b90764e	segmentation: adds xxtiny, but unsure if it's small enough	2022-10-11 19:22:37 +01:00
Starbeamrainbowlabs	f4a2c742d9	typo	2022-10-11 19:19:23 +01:00
Starbeamrainbowlabs	11f91a7cf4	train: add --arch; default to convnext_i_xtiny	2022-10-11 19:18:01 +01:00
Starbeamrainbowlabs	5666c5a0d9	typo	2022-10-10 18:12:51 +01:00
Starbeamrainbowlabs	131c0a0a5b	pretrain-predict: create dir if not exists	2022-10-10 18:00:55 +01:00
Starbeamrainbowlabs	deede32241	slurm-pretrain: limit memory usage	2022-10-10 17:45:29 +01:00
Starbeamrainbowlabs	13a8f3f511	pretrain-predict: only queue pretrain-plot if I we output jsonl	2022-10-10 17:11:10 +01:00
Starbeamrainbowlabs	ffcb2e3735	pretrain-predict: queue for the actual input	2022-10-10 16:53:28 +01:00
Starbeamrainbowlabs	f883986eaa	Bugfix: modeset to enable TFRecordWriter instead of bare handle	2022-10-06 20:07:59 +01:00
Starbeamrainbowlabs	e9a8e2eb57	fixup	2022-10-06 19:23:31 +01:00
Starbeamrainbowlabs	9f3ae96894	finish wiring for --water-size	2022-10-06 19:21:50 +01:00
Starbeamrainbowlabs	5dac70aa08	typo	2022-10-06 19:17:03 +01:00
Starbeamrainbowlabs	2960d3b645	exception → warning	2022-10-06 18:26:40 +01:00
Starbeamrainbowlabs	0ee6703c1e	Add todo and comment	2022-10-03 19:06:56 +01:00
Starbeamrainbowlabs	2b182214ea	typo	2022-10-03 17:53:10 +01:00
Starbeamrainbowlabs	92c380bff5	fiddle with Conv2DTranspose ... you need to set the `stride` argument to actually get it to upscale..... :P	2022-10-03 17:51:41 +01:00
Starbeamrainbowlabs	d544553800	fixup	2022-10-03 17:33:06 +01:00
Starbeamrainbowlabs	058e3b6248	model_segmentation: cast float → int	2022-10-03 17:31:36 +01:00
Starbeamrainbowlabs	04e5ae0c45	model_segmentation: redo reshape ... much cheese was applied :P	2022-10-03 17:27:52 +01:00
Starbeamrainbowlabs	deffe69202	typo	2022-10-03 16:59:36 +01:00
Starbeamrainbowlabs	fc6d2dabc9	Upscale first, THEN convnext...	2022-10-03 16:38:43 +01:00
Starbeamrainbowlabs	6a0790ff50	convnext_inverse: add returns; change ordering	2022-10-03 16:32:09 +01:00
Starbeamrainbowlabs	fe813cb46d	slurm predict: fix plotting subcall	2022-10-03 16:03:26 +01:00
Starbeamrainbowlabs	e51087d0a9	add reshape layer	2022-09-28 18:22:48 +01:00
Starbeamrainbowlabs	a336cdee90	and continues	2022-09-28 18:18:10 +01:00
Starbeamrainbowlabs	de47a883d9	missing units	2022-09-28 18:17:22 +01:00
Starbeamrainbowlabs	b5e08f92fe	the long night continues	2022-09-28 18:14:09 +01:00
Starbeamrainbowlabs	dc159ecfdb	and again	2022-09-28 18:11:46 +01:00
Starbeamrainbowlabs	4cf0485e32	fixup... again	2022-09-28 18:10:11 +01:00
Starbeamrainbowlabs	030d8710b6	fixup	2022-09-28 18:08:31 +01:00
Starbeamrainbowlabs	4ee7f2a0d6	add water thresholding	2022-09-28 18:07:26 +01:00
Starbeamrainbowlabs	404dc30f08	and again	2022-09-28 17:39:09 +01:00
Starbeamrainbowlabs	4cd8fc6ded	segmentation: param name fix	2022-09-28 17:37:42 +01:00
Starbeamrainbowlabs	41ba980d69	segmentationP implement dataset parser	2022-09-28 17:19:21 +01:00
Starbeamrainbowlabs	d618e6f8d7	pretrain-predict: params.json → metadata.jsonl	2022-09-28 16:35:22 +01:00
Starbeamrainbowlabs	e9e6139c7a	typo	2022-09-28 16:28:18 +01:00
Starbeamrainbowlabs	3dee3d8908	update cli help	2022-09-28 16:23:47 +01:00
Starbeamrainbowlabs	b836f7f70c	again	2022-09-27 19:06:41 +01:00
Starbeamrainbowlabs	52dff130dd	slurm pretraian predict: moar logging	2022-09-27 19:05:54 +01:00
Starbeamrainbowlabs	2d24174e0a	slurm pretrain predict: add $OUTPUT	2022-09-27 18:58:02 +01:00
Starbeamrainbowlabs	7d0e3913ae	fix logging	2022-09-27 18:51:58 +01:00
Starbeamrainbowlabs	d765b3b14e	fix crash	2022-09-27 18:43:43 +01:00
Starbeamrainbowlabs	2cd59a01a5	slurm-pretrain-plot: add ARGS	2022-09-27 18:41:35 +01:00
Starbeamrainbowlabs	f4d1d1d77e	just wh	2022-09-27 18:25:45 +01:00
Starbeamrainbowlabs	4c24d69ae6	$d → +d	2022-09-27 18:17:07 +01:00
Starbeamrainbowlabs	cdb19b4d9f	fixup	2022-09-27 18:13:21 +01:00
Starbeamrainbowlabs	c4d3c16873	add some logging	2022-09-27 18:10:58 +01:00
Starbeamrainbowlabs	3772c3227e	fixup	2022-09-27 17:57:21 +01:00
Starbeamrainbowlabs	dbfa45a016	write params.json properly	2022-09-27 17:49:54 +01:00
Starbeamrainbowlabs	a5455dc22a	fix import	2022-09-27 17:41:24 +01:00
Starbeamrainbowlabs	d6ff3fb2ce	pretrain_predict fix write mode	2022-09-27 17:38:12 +01:00
Starbeamrainbowlabs	f95fd8f9e4	pretrain-predict: add .tfrecord output function	2022-09-27 16:59:31 +01:00
Starbeamrainbowlabs	30b8dd063e	fixup	2022-09-27 15:54:37 +01:00
Starbeamrainbowlabs	3cf99587e4	Contraster: add gamma layer to load_model	2022-09-27 15:53:52 +01:00
Starbeamrainbowlabs	9e9852d066	UMAP: 100k random ... no labels.	2022-09-27 15:52:45 +01:00
Starbeamrainbowlabs	58c65bdc86	slurm: allow predictions on gpu	2022-09-23 19:21:57 +01:00
Starbeamrainbowlabs	d59de41ebb	embeddings: change title rendering; make even moar widererer ... We need to see that parallel coordinates plot in detail	2022-09-23 18:56:39 +01:00
Starbeamrainbowlabs	b4ddb24589	slurm plot: compute → highmem	2022-09-22 18:27:58 +01:00
Starbeamrainbowlabs	df12470e78	flip conda and time ... hopefully we can capture the exit code this way	2022-09-21 14:40:28 +01:00
Starbeamrainbowlabs	5252a81238	vis: don't call add_subplot	2022-09-20 19:06:21 +01:00
Starbeamrainbowlabs	32bb55652b	slurm predict: autoqueue UMAP plot	2022-09-16 19:36:57 +01:00
Starbeamrainbowlabs	24c5263bf0	slurm predict-plot: fixup	2022-09-16 19:27:26 +01:00
Starbeamrainbowlabs	7e9c119b04	slurm: fix job name	2022-09-16 19:20:59 +01:00
Starbeamrainbowlabs	1574529704	slurm pretrain-predict: move to compute; make exclusive just in case ... also shortent o 3 days	2022-09-16 19:17:42 +01:00
Starbeamrainbowlabs	d7f5958af0	slurm: write new job files	2022-09-16 19:00:43 +01:00