Freeze/crash when transferring large content to BTT Mobile

I've been experimenting with (what I think is) the latest version of BTT Mobile, and I'm pleased to see that the floating webviews now have the javascript functionality enabled to allow bidirectional communication from BTT Mobile to the host Mac, and from the host Mac back to BTT Mobile. However, I've encountered some freezing / crashing behaviour from BTT Mobile when I attempt to transfer large content from the host Mac to a floating webview on BTT Mobile.

Here's the setup: in my floating webview on BTT Mobile, I have a simple grid display to experiment with transferring content from my Mac when I press a button. On my Mac, I use the command line to put base64 encoded images, video, or audio into the clipboard (e.g., base64 -i image.png | pbcopy), and then use native BTT methods in the floating webview to transfer the base64 encoded data, and then dynamically generate elements with that data as their content. (At this stage, there's no error checking to make sure that the content is of the right type - I have to make sure that I don't make a mistake when adding stuff to the clipboard.)

Here's some of the html from the floating webview:

<div style='display: grid; grid-template-columns: auto 1fr; row-gap: 6pt; align-items: start; column-gap: 6pt;'>
	<button onclick='addElement("img")'>Add image</button><span id='my-img'></span>
	<button onclick='addElement("video")'>Add video</button><span id='my-video'></span>
	<button onclick='addElement("audio")'>Add audio</button><span id='my-audio'></span>
</div>

The addElement() function is defined as follows (the floating webview loads jQuery to help with DOM manipulation, as well as shoelace.style web components for the sl-spinner element):

async function addElement(type) {
	let data;				

	switch(type) {
		case 'img':
			$('#my-img').html("<sl-spinner></sl-spinner>");
			data = await get_clipboard_content();
			$('#my-img').html(`<img src="data:image/png;base64,${data}" style="height: 10cm;">`);
			break;
		case 'video':
			$('#my-video').html("<sl-spinner></sl-spinner>");
			data = await get_clipboard_content();
			$('#my-video').html(`<video src="data:video/mp4;base64,${data}" controls style="width: 100%;"></video>`);
			break;
		case 'audio':
			$('#my-audio').html("<sl-spinner></sl-spinner>");
			data = await get_clipboard_content();
			$('#my-audio').html(`<audio src="data:audio/mp4;base64,${data}" controls style="width: 100%;"></audio>`);
			break;
	}
}

This works well for image and audio files, because (I assume) the base64 data is not too large to cause problems with the wireless transfer from the Mac to BTT Mobile, as the screenshot below shows:

However, if I try to do this with a video file, say a 22MB scaled-down version of a Rolling Stones music video, it only succeeds about 30% of the time... most often I get a spinning wheel indicating that BTT Mobile is attempting to reconnect, which never succeeds (see screenshot below). Once this occurs, I often have to kill BTT Mobile and BTT on my Mac to get things working smoothly again.

Any suggestions on what I could do to help avoid this would be most appreciated!

In addition, is this the best way to transfer content from my mac into floating webviews on BTT Mobile? Should I be using BTT Variables? (I'm not sure if BTT Variables have an upper-limit on how much data they can hold.) I ended up using this solution because I found it too difficult to put binary data on the clipboard, whereas base64 encoded data was trivial.

Many thanks!

Jason

Variables are indeed not well suited for this - on the one hand because their storage is very simple and not optimized for size at all, on the other hand because they transfer kind of synchronously and are not made for such large data.

Currently the only supported way that should work (but I haven't really tried that with large data either) is to use the presetfile url scheme like this:

<img src="presetfile://presetname/somefile.jpeg" />

This however can only access files located in your preset's data folder, to open that folder:

Hi Andreas,

Thanks for the quick reply!

I've tried including elements from the preset folder using the following syntax:

<video src="presetfile://Default/video.mp4" controls style="width: 100%;"></video>

but that doesn’t work. The video fails to load and generally causes BTT Mobile to crash. If I do the same with a smaller audio clip, I get “error” displayed when the audio player attempts to load the file (but no crash).

Interestingly, there’s no problem loading images with the presetfile:// syntax. It also seems that normal relative paths work as well, when used inside the HTML file defining the floating web view shown on BTT Mobile, but again relative paths don’t work for video or audio.

I haven’t tried embedding PDFs in a web view, yet, but I would bet they would work fine as long as they aren’t too large. (The PGF/TiKZ user manual will probably break things, though. :grin:)

J

Just for completeness, I wanted to note that after a little more experimenting, I’ve confirmed that the reason the embedding effort failed is due to some built-in limit on the length of string transfers to BTT Mobile when calling get_clipboard_content() from JavaScript within a BTT Mobile webview.

I was able to establish this by uploading the base64-encoded video data to my web server and replacing the relevant lines in the function addElement() with the following:

case 'video':
	$('#my-video').html("<sl-spinner></sl-spinner>");
	//data = await get_clipboard_content();
	const response = await fetch('...URL for the link...');
	data = await response.text();
	$('#my-video').html(`<video src="data:video/mp4;base64,${data}" controls style="width: 100%;"></video>`);
break;

On one hand, this is good news because it means I'm not hitting a JavaScript limit which prevents these kind of dynamic elements from being generated and used within BTT Mobile webviews. (The video data is 30MB and BTT Mobile handles that fine.) So in principle I should be able to do this dynamically using get_clipboard_contents() if I just chunked the base64 data and transferred it in stages. But it would be helpful to know what the limit on such transfers is.

One final note: I don't seem to be able to embed video or audio elements in the webview using src='presetfile://Default/...' at all. So the only way to include such elements in a BTT Mobile webview is either to host the content on a separate server or to embed it dynamically using the method described above.

Many thanks — and apologies for boring on about what is undoubtedly a very niche use-case!

Jason

Thanks for the details! I'll improve this soon.