645 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Starbeamrainbowlabs	090ab78457	dlr: fix (another) crash	2024-11-14 21:53:59 +00:00
Starbeamrainbowlabs	7be2222207	dlr: import ...as env	2024-11-14 21:52:41 +00:00
Starbeamrainbowlabs	a7ab5ee341	slurm/dlr: cpu cores 14→9	2024-11-14 21:49:54 +00:00
Starbeamrainbowlabs	e83d901779	env.py: fix crash	2024-11-14 21:49:03 +00:00
Starbeamrainbowlabs	2b69d2c4f2	slurm/dlr: correct logging msg	2024-11-14 21:37:11 +00:00
Starbeamrainbowlabs	bd2c6b1c3f	slurm/dlr: don't runmodule load .... on csgpu cluster	2024-11-14 21:34:04 +00:00
Starbeamrainbowlabs	4ac7082754	write commit info to file in DIR_OUTPUT	2024-11-14 21:17:53 +00:00
Starbeamrainbowlabs	8befef5fc1	slurm/dlr: logging ... it Should™ work now? TODO test this!	2024-11-14 19:55:59 +00:00
Starbeamrainbowlabs	c82e4bf5ac	Merge branch 'main' of github.com:sbrl/research-rainfallradar	2024-11-14 19:09:39 +00:00
Starbeamrainbowlabs	daa923a88b	pyproject.toml: typo	2024-11-14 19:09:26 +00:00
Starbeamrainbowlabs	d0dc1b4280	.gitignore: ignore some more backup files: draw.io, .bak.png	2024-11-08 22:05:13 +00:00
Starbeamrainbowlabs	04ea305b70	dlr/slurm: implement USE_CONDA, module command opt-support	2024-11-08 21:47:55 +00:00
Starbeamrainbowlabs	f94f777f0a	dlr: make cv2 import optional	2024-11-01 20:40:55 +00:00
Starbeamrainbowlabs	05fd955067	README: note about RANDSEED	2024-08-30 19:02:18 +01:00
Starbeamrainbowlabs	e908e4b693	dlr: Note about RANDSEED	2024-08-30 18:50:26 +01:00
Starbeamrainbowlabs	f46a86c9a4	README: Document SPLIT_VALIDATE, SPLIT_TEST	2024-08-30 18:49:38 +01:00
Starbeamrainbowlabs	0761651ccf	tvt: implement CallbackExtraValidation, which allows for a third split ... it should tie into Tensorflow's logging just fine so long as it's the first callback in the queue. ***** TEST SCRIPT ***** model = tf.keras.Sequential([ tf.keras.layers.Dense(64, activation='relu', input_shape=(10,)), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dense(1) ]) model.compile(optimizer='adam', loss='mse', metrics=['mae']) X = np.random.random((100, 10)) y = np.random.random((100, 1)) split = 80 X_train, X_val = X[:split], X[split:] y_train, y_val = y[:split], y[split:] train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(10) val_dataset = tf.data.Dataset.from_tensor_slices((X_val, y_val)).batch(10) history = model.fit(train_dataset, epochs=10, validation_data=val_dataset, callbacks=[ CallbackExtraValidation({ "test": val_dataset }, verbose=0), tf.keras.callbacks.CSVLogger("/dev/stdout", separator="\t") ], verbose=0 ) print(f"DEBUG history {history}")	2024-08-30 18:07:17 +01:00
Starbeamrainbowlabs	b5310304bd	wrangler: update dependencies	2024-08-29 19:38:02 +01:00
Starbeamrainbowlabs	e5f6e6394f	Implement initial UNTESTED support for split_validation and split_test	2024-08-29 19:33:40 +01:00
Starbeamrainbowlabs	b2b96ab636	replace tensorflow_addons with keras_cv	2024-08-29 19:32:54 +01:00
Starbeamrainbowlabs	2d73d84e20	uv pip sync doesn't install dependencies of packages	2024-08-29 19:31:42 +01:00
Starbeamrainbowlabs	b31485f7a1	add pyproject.toml	2024-08-29 18:58:57 +01:00
Starbeamrainbowlabs	61ee2326cf	licence change	2024-08-29 18:36:11 +01:00
Starbeamrainbowlabs	18fd4b5d4f	Correct github language bar; Jupyter notebook are not part of the main project	2024-08-29 18:32:28 +01:00
Starbeamrainbowlabs	c98c42fa7e	dataset_mono: Implement validate_percentage + test_percentage support ... This removes the train_percentage argument TODO: map this forwards to enable support in deeplabv3_plus_test_rainfall ...thinking about it, it's really not a test now, is it? Updating the filename would be such a /hassle/ though....	2024-08-29 16:44:19 +01:00
Starbeamrainbowlabs	5d62e3cee8	Implement env from PhD-smflooding-scene	2024-08-29 16:43:29 +01:00
Starbeamrainbowlabs	a75d4f5d79	dataset_mono: add todo	2024-08-02 19:48:29 +01:00
Starbeamrainbowlabs	e15f6cd954	Update README w/proper paper link	2024-06-21 16:03:58 +01:00
Starbeamrainbowlabs	d22dd6ff18	README: more tidy up	2023-11-30 17:08:50 +00:00
Starbeamrainbowlabs	1cbb241786	README: tidy up	2023-11-30 17:02:08 +00:00
Starbeamrainbowlabs	844d8e6dd4	rw: update dependencies	2023-11-30 17:01:13 +00:00
Starbeamrainbowlabs	c5be527ffb	rw/README: fill out	2023-11-30 16:59:52 +00:00
Starbeamrainbowlabs	4626549352	rw/recompress: clarify subcommand description	2023-11-30 16:59:45 +00:00
Starbeamrainbowlabs	18195c2d15	rw: mark as 'private' in package.json to avoid accidental publication	2023-11-30 16:59:31 +00:00
Starbeamrainbowlabs	002203d20b	vscodium/settings: spellings	2023-11-30 16:59:06 +00:00
Starbeamrainbowlabs	294fde7c82	rw; add requirements.txt	2023-11-30 16:45:04 +00:00
Starbeamrainbowlabs	92fc34ebb1	rw/child_process:json2tfrecord: Add RAINFALL_MAX_NUMBER env var ... use use you'd need to export RAINFALL_MAX_NUMBER=some_integer_value, since it is used by a subprocess rather than the main Node.js process itself	2023-11-30 16:44:43 +00:00
Starbeamrainbowlabs	0f9f185983	dlr: add PARALLEL_READS env var, update docs	2023-11-30 16:33:22 +00:00
Starbeamrainbowlabs	60674fb6b3	README: finish filling out	2023-11-30 16:32:46 +00:00
Starbeamrainbowlabs	13ae4df725	vscodium/settings: spellings	2023-11-30 16:32:17 +00:00
Starbeamrainbowlabs	df621fd7d2	README: Continue filling out, but we're not there yet.	2023-11-29 17:31:47 +00:00
Starbeamrainbowlabs	3eec01a28c	README: update image	2023-11-29 15:16:01 +00:00
Starbeamrainbowlabs	a1a2b105e6	README: Add under construction notice	2023-11-29 15:15:07 +00:00
Starbeamrainbowlabs	1d8807793f	README: start expanding, buti ts not finished yet	2023-11-28 16:46:02 +00:00
Starbeamrainbowlabs	e14fa275ab	rr de: rerun to recaalculate ... these values are accurate to our dataset now	2023-11-23 18:35:39 +00:00
Starbeamrainbowlabs	76b2956703	rr de: update to dataset_mono_predict	2023-11-03 15:48:53 +00:00
Starbeamrainbowlabs	cbb916327b	vscode → settings.json: spellings	2023-10-26 17:12:46 +01:00
Starbeamrainbowlabs	e44b5533b1	plot_metrics_multi: add option to plot train/val separately	2023-07-20 15:49:09 +01:00
Starbeamrainbowlabs	b5b26d980b	dlr: print filepaths ... we need to know which is which with this seed so we can visualise for the paper	2023-07-13 19:58:38 +01:00
Starbeamrainbowlabs	d7fdf8f038	README: fix diagram	2023-07-06 15:31:13 +01:00