From 343008c0af65f56474b61a2fdc2503fd4fb923ef Mon Sep 17 00:00:00 2001 From: "A.Olokhtonov" Date: Sat, 15 Apr 2023 17:13:49 +0300 Subject: [PATCH] f32 coordinates --- client/client_recv.js | 350 +++++++++++++++++++++++++++++ client/{send.js => client_send.js} | 88 +++++--- client/index.html | 116 ++++++++-- client/index.js | 38 ---- client/math.js | 2 +- client/old_index.html | 44 ++++ client/recv.js | 345 ---------------------------- client/webgl.html | 110 --------- client/webgl.js | 103 ++++++++- client/webgl_geometry.js | 92 ++++++-- client/webgl_listeners.js | 60 ++--- client/websocket.js | 32 +-- server/deserializer.js | 33 ++- server/recv.js | 1 - server/send.js | 4 +- server/serializer.js | 15 +- 16 files changed, 793 insertions(+), 640 deletions(-) create mode 100644 client/client_recv.js rename client/{send.js => client_send.js} (66%) create mode 100644 client/old_index.html delete mode 100644 client/recv.js delete mode 100644 client/webgl.html diff --git a/client/client_recv.js b/client/client_recv.js new file mode 100644 index 0000000..8a19afc --- /dev/null +++ b/client/client_recv.js @@ -0,0 +1,350 @@ +function deserializer_create(buffer, dataview) { + return { + 'offset': 0, + 'size': buffer.byteLength, + 'buffer': buffer, + 'view': dataview, + 'strview': new Uint8Array(buffer), + }; +} + +function des_u8(d) { + const value = d.view.getUint8(d.offset); + d.offset += 1; + return value; +} + +function des_u16(d) { + const value = d.view.getUint16(d.offset, true); + d.offset += 2; + return value; +} + +function des_u32(d) { + const value = d.view.getUint32(d.offset, true); + d.offset += 4; + return value; +} + +function des_f32(d) { + const value = d.view.getFloat32(d.offset, true); + d.offset += 4; + return value; +} + +function des_f32array(d, count) { + const result = []; + + for (let i = 0; i < count; ++i) { + const item = d.view.getFloat32(d.offset, true); + d.offset += 4; + result.push(item); + } + + return result; +} + +function des_event(d) { + const event = {}; + + event.type = des_u8(d); + event.user_id = des_u32(d); + + switch (event.type) { + case EVENT.PREDRAW: { + event.x = des_f32(d); + event.y = des_f32(d); + break; + } + + case EVENT.STROKE: { + const stroke_id = des_u32(d); + const point_count = des_u16(d); + const width = des_u16(d); + const color = des_u32(d); + const coords = des_f32array(d, point_count * 2); + + event.stroke_id = stroke_id; + event.points = []; + + for (let i = 0; i < point_count; ++i) { + const x = coords[2 * i + 0]; + const y = coords[2 * i + 1]; + event.points.push({'x': x, 'y': y}); + } + + event.color = color; + event.width = width; + + break; + } + + case EVENT.IMAGE: + case EVENT.IMAGE_MOVE: { + event.image_id = des_u32(d); + event.x = des_f32(d); + event.y = des_f32(d); + break; + } + + case EVENT.UNDO: + case EVENT.REDO: { + break; + } + + case EVENT.ERASER: { + event.stroke_id = des_u32(d); + break; + } + + default: { + console.error('fuck'); + } + } + + return event; +} + +function bitmap_bbox(event) { + const bbox = { + 'xmin': event.x, + 'xmax': event.x + event.bitmap.width, + 'ymin': event.y, + 'ymax': event.y + event.bitmap.height + }; + + return bbox; +} + +async function handle_event(state, context, event, relax = false) { + if (config.debug_print) console.debug(`event type ${event.type} from user ${event.user_id}`); + + switch (event.type) { + case EVENT.STROKE: { + if (event.user_id != state.me) { + clear_dynamic_stroke(state, context, event.user_id); + } + + add_static_stroke(state, context, event, relax); + + break; + } + + case EVENT.UNDO: { + console.error('todo'); + // for (let i = state.events.length - 1; i >=0; --i) { + // const other_event = state.events[i]; + + // // Users can only undo their own, undeleted (not already undone) events + // if (other_event.user_id === event.user_id && !other_event.deleted) { + // if (other_event.type === EVENT.STROKE) { + // other_event.deleted = true; + // const stats = stroke_stats(other_event.points, state.cursor.width); + // redraw_region(stats.bbox); + // break; + // } else if (other_event.type === EVENT.IMAGE) { + // other_event.deleted = true; + // const item = document.querySelector(`img[data-image-id="${other_event.image_id}"]`); + // if (item) item.remove(); + // break; + // } else if (other_event.type === EVENT.ERASER) { + // other_event.deleted = true; + // const erased = find_stroke_backwards(other_event.stroke_id); + // if (erased) { + // erased.deleted = false; + // const stats = stroke_stats(erased.points, state.cursor.width); + // redraw_region(stats.bbox); + // } + // break; + // } else if (other_event.type === EVENT.IMAGE_MOVE) { + // const item = document.querySelector(`img[data-image-id="${other_event.image_id}"]`); + + // const ix = state.images[other_event.image_id].x -= other_event.x; + // const iy = state.images[other_event.image_id].y -= other_event.y; + + // item.style.transform = `translate(${ix}px, ${iy}px)`; + + // break; + // } + // } + // } + + break; + } + + case EVENT.IMAGE: { + console.error('todo'); + // const url = config.image_url + event.image_id; + // const item = document.createElement('img'); + + // item.classList.add('floating-image'); + // item.style['z-index'] = state.events.length; + // item.setAttribute('data-image-id', event.image_id); + // item.setAttribute('src', url); + // item.style.transform = `translate(${event.x}px, ${event.y}px)`; + + // elements.images.appendChild(item); + // state.images[event.image_id] = { + // 'x': event.x, 'y': event.y + // }; + // const r = await fetch(config.image_url + event.image_id); + // const blob = await r.blob(); + // const bitmap = await createImageBitmap(blob); + // event.bitmap = bitmap; + // const bbox = bitmap_bbox(event); + // state.ctx0.drawImage(bitmap, bbox.xmin, bbox.ymin); + + break; + } + + case EVENT.IMAGE_MOVE: { + console.error('todo'); + // // Already moved due to local prediction + // if (event.user_id !== state.me.id) { + // const image_id = event.image_id; + // const item = document.querySelector(`.floating-image[data-image-id="${image_id}"]`); + + // const ix = state.images[event.image_id].x += event.x; + // const iy = state.images[event.image_id].y += event.y; + + // if (item) { + // item.style.transform = `translate(${ix}px, ${iy}px)`; + // } + // } + + break; + } + + case EVENT.ERASER: { + console.error('todo'); + // if (event.deleted) { + // break; + // } + + // for (const other_event of state.events) { + // if (other_event.type === EVENT.STROKE && other_event.stroke_id === event.stroke_id) { + // // Might already be deleted because of local prediction + // if (!other_event.deleted) { + // other_event.deleted = true; + // const stats = stroke_stats(other_event.points, state.cursor.width); + // redraw_region(stats.bbox); + // } + // break; + // } + // } + break; + } + + default: { + console.error('fuck'); + } + } +} + +async function handle_message(state, context, d) { + const message_type = des_u8(d); + let do_draw = false; + + if (config.debug_print) console.debug(message_type); + + switch (message_type) { + case MESSAGE.JOIN: + case MESSAGE.INIT: { + state.me = des_u32(d); + state.server_lsn = des_u32(d); + + if (state.server_lsn > state.lsn) { + // Server knows something that we don't + state.lsn = state.server_lsn; + } + + if (message_type === MESSAGE.JOIN) { + localStorage.setItem('sessionId', des_u32(d)); + if (config.debug_print) console.debug('join in'); + } else { + if (config.debug_print) console.debug('init in'); + } + + const event_count = des_u32(d); + + if (config.debug_print) console.debug(`${event_count} events in init`); + + state.events.length = 0; + + for (let i = 0; i < event_count; ++i) { + const event = des_event(d); + await handle_event(state, context, event, true); + state.events.push(event); + } + + recompute_static_data(context); + + do_draw = true; + + send_ack(event_count); + + sync_queue(state); + + break; + } + + case MESSAGE.FIRE: { + const user_id = des_u32(d); + const predraw_event = des_event(d); + + update_dynamic_stroke(state, context, user_id, {'x': predraw_event.x, 'y': predraw_event.y}); + + do_draw = true; + + break; + } + + case MESSAGE.ACK: { + const lsn = des_u32(d); + + if (config.debug_print) console.debug(`ack ${lsn} in`); + + if (lsn > state.server_lsn) { + // ACKs may arrive out of order + state.server_lsn = lsn; + } + + break; + } + + case MESSAGE.SYN: { + const sn = des_u32(d); + const count = des_u32(d); + + const we_expect = sn - state.sn; + const first = count - we_expect; + + if (config.debug_print) console.debug(`syn ${sn} in`); + + for (let i = 0; i < count; ++i) { + const event = des_event(d); + if (i >= first) { + handle_event(state, context, event); + state.events.push(event); + } + } + + do_draw = true; + + state.sn = sn; + + send_ack(sn); // await? + + break; + } + + default: { + console.error('fuck'); + return; + } + } + + if (do_draw) { + window.requestAnimationFrame(() => draw(state, context)); + } +} diff --git a/client/send.js b/client/client_send.js similarity index 66% rename from client/send.js rename to client/client_send.js index 85002b6..3d793bb 100644 --- a/client/send.js +++ b/client/client_send.js @@ -19,19 +19,29 @@ function ser_u16(s, value) { s.offset += 2; } +function ser_f32(s, value) { + s.view.setFloat32(s.offset, value, true); + s.offset += 4; +} + function ser_u32(s, value) { s.view.setUint32(s.offset, value, true); s.offset += 4; } +function ser_align(s, to) { + while (s.offset % to != 0) { + s.offset++; + } +} + function ser_event(s, event) { ser_u8(s, event.type); - ser_u8(s, 0); // padding for 16bit alignment - + switch (event.type) { case EVENT.PREDRAW: { - ser_u16(s, event.x); - ser_u16(s, event.y); + ser_f32(s, event.x); + ser_f32(s, event.y); break; } @@ -42,9 +52,11 @@ function ser_event(s, event) { if (config.debug_print) console.debug('original', event.points); + ser_align(s, 4); + for (const point of event.points) { - ser_u16(s, point.x); - ser_u16(s, point.y); + ser_f32(s, point.x); + ser_f32(s, point.y); } break; @@ -54,8 +66,8 @@ function ser_event(s, event) { case EVENT.IMAGE_MOVE: { const image_id = parseInt(event.image_id); ser_u32(s, image_id); - ser_u16(s, event.x); - ser_u16(s, event.y); + ser_f32(s, event.x); + ser_f32(s, event.y); break; } @@ -90,39 +102,38 @@ async function send_ack(sn) { } } -async function sync_queue() { +async function sync_queue(state) { if (ws === null) { if (config.debug_print) console.debug('socket has closed, stopping SYNs'); return; } - let size = 1 + 1 + 4 + 4; // opcode + lsn + event count - let count = storage.lsn - storage.server_lsn; + let size = 1 + 3 + 4 + 4; // opcode + lsn + event count + let count = state.lsn - state.server_lsn; if (count === 0) { if (config.debug_print) console.debug('server ACKed all events, clearing queue'); - storage.queue.length = 0; + state.queue.length = 0; return; } for (let i = count - 1; i >= 0; --i) { - const event = storage.queue[storage.queue.length - 1 - i]; + const event = state.queue[state.queue.length - 1 - i]; size += event_size(event); } const s = serializer_create(size); ser_u8(s, MESSAGE.SYN); - ser_u8(s, 0); // padding for 16bit alignment - ser_u32(s, storage.lsn); + ser_u32(s, state.lsn); ser_u32(s, count); for (let i = count - 1; i >= 0; --i) { - const event = storage.queue[storage.queue.length - 1 - i]; + const event = state.queue[state.queue.length - 1 - i]; ser_event(s, event); } - if (config.debug_print) console.debug(`syn ${storage.lsn} out`); + if (config.debug_print) console.debug(`syn ${state.lsn} out`); try { if (ws) await ws.send(s.buffer); @@ -130,19 +141,17 @@ async function sync_queue() { ws.close(); } - setTimeout(sync_queue, config.sync_timeout); + setTimeout(() => sync_queue(state), config.sync_timeout); } -function push_event(event) { - storage.lsn += 1; +function push_event(state, event) { + state.lsn += 1; switch (event.type) { case EVENT.STROKE: { - const points = process_stroke(event.points); - - storage.queue.push({ + state.queue.push({ 'type': EVENT.STROKE, - 'points': points, + 'points': event.points, 'width': event.width, 'color': event.color, }); @@ -152,7 +161,7 @@ function push_event(event) { case EVENT.RULER: { event.type = EVENT.STROKE; - storage.queue.push(event); + state.queue.push(event); break; } @@ -161,7 +170,7 @@ function push_event(event) { case EVENT.IMAGE_MOVE: case EVENT.UNDO: case EVENT.REDO: { - storage.queue.push(event); + state.queue.push(event); break; } @@ -172,18 +181,18 @@ function push_event(event) { } // Queue an event and initialize repated sends until ACKed -function queue_event(event, skip = false) { - push_event(event); +function queue_event(state, event, skip = false) { + push_event(state, event); if (skip) { return; } - if (storage.timers.queue_sync) { - clearTimeout(storage.timers.queue_sync); + if (state.timers.queue_sync) { + clearTimeout(state.timers.queue_sync); } - sync_queue(); + sync_queue(state); } // Fire and forget. Doesn't do anything if we are offline @@ -198,4 +207,21 @@ async function fire_event(event) { } catch(e) { ws.close(); } +} + +function predraw_event(x, y) { + return { + 'type': EVENT.PREDRAW, + 'x': x, + 'y': y + }; +} + +function stroke_event(state) { + return { + 'type': EVENT.STROKE, + 'points': state.current_strokes[state.me].points, + 'width': state.current_strokes[state.me].width, + 'color': state.current_strokes[state.me].color, + }; } \ No newline at end of file diff --git a/client/index.html b/client/index.html index 17f41ac..283ee99 100644 --- a/client/index.html +++ b/client/index.html @@ -5,23 +5,101 @@ Desk - - - - - - - - + - - + + + + + + + + + + + + -
- - -
+
@@ -32,13 +110,5 @@
- -
- - - -
- -
- + \ No newline at end of file diff --git a/client/index.js b/client/index.js index 408ac93..9c9c0a8 100644 --- a/client/index.js +++ b/client/index.js @@ -104,44 +104,6 @@ const elements = { 'active_image': null, }; -function event_size(event) { - let size = 1 + 1; // type + padding - - switch (event.type) { - case EVENT.PREDRAW: { - size += 2 * 2; - break; - } - - case EVENT.STROKE: { - size += 4 + 2 + 2 + 4 + event.points.length * 2 * 2; // u32 stroke id + u16 (count) + u16 (width) + u32 (color + count * (u16, u16) points - break; - } - - case EVENT.UNDO: - case EVENT.REDO: { - break; - } - - case EVENT.IMAGE: - case EVENT.IMAGE_MOVE: { - size += 4 + 2 + 2; // file id + x + y - break; - } - - case EVENT.ERASER: { - size += 4; // stroke id - break; - } - - default: { - console.error('fuck'); - } - } - - return size; -} - function move_canvas() { elements.canvas0.style.transform = `translate(${-storage.canvas.offset_x}px, ${-storage.canvas.offset_y}px) scale(${storage.canvas.zoom})`; elements.canvas1.style.transform = `translate(${-storage.canvas.offset_x}px, ${-storage.canvas.offset_y}px) scale(${storage.canvas.zoom})`; diff --git a/client/math.js b/client/math.js index d542a75..2d38e62 100644 --- a/client/math.js +++ b/client/math.js @@ -12,7 +12,7 @@ function point_right_of_line(a, b, p) { } function rdp_find_max(points, start, end) { - const EPS = 0.5; + const EPS = 0.5; // TODO: base this on zoom (and/or "speed") let result = -1; let max_dist = 0; diff --git a/client/old_index.html b/client/old_index.html new file mode 100644 index 0000000..17f41ac --- /dev/null +++ b/client/old_index.html @@ -0,0 +1,44 @@ + + + + + Desk + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+
+
+
+ +
+
+ +
+ + + +
+ +
+ + diff --git a/client/recv.js b/client/recv.js deleted file mode 100644 index e69c255..0000000 --- a/client/recv.js +++ /dev/null @@ -1,345 +0,0 @@ -function deserializer_create(buffer, dataview) { - return { - 'offset': 0, - 'size': buffer.byteLength, - 'buffer': buffer, - 'view': dataview, - 'strview': new Uint8Array(buffer), - }; -} - -function des_u8(d) { - const value = d.view.getUint8(d.offset); - d.offset += 1; - return value; -} - -function des_u16(d) { - const value = d.view.getUint16(d.offset, true); - d.offset += 2; - return value; -} - -function des_s16(d) { - const value = d.view.getInt16(d.offset, true); - d.offset += 2; - return value; -} - -function des_u32(d) { - const value = d.view.getUint32(d.offset, true); - d.offset += 4; - return value; -} - -function des_u16array(d, count) { - const result = []; - - for (let i = 0; i < count; ++i) { - const item = d.view.getUint16(d.offset, true); - d.offset += 2; - result.push(item); - } - - return result; -} - -function des_event(d) { - const event = {}; - - event.type = des_u8(d); - event.user_id = des_u32(d); - - switch (event.type) { - case EVENT.PREDRAW: { - event.x = des_u16(d); - event.y = des_u16(d); - break; - } - - case EVENT.STROKE: { - const stroke_id = des_u32(d); - const point_count = des_u16(d); - const width = des_u16(d); - const color = des_u32(d); - const coords = des_u16array(d, point_count * 2); - - event.stroke_id = stroke_id; - event.points = []; - - for (let i = 0; i < point_count; ++i) { - const x = coords[2 * i + 0]; - const y = coords[2 * i + 1]; - event.points.push({'x': x, 'y': y}); - } - - event.color = color; - event.width = width; - - break; - } - - case EVENT.IMAGE: - case EVENT.IMAGE_MOVE: { - event.image_id = des_u32(d); - event.x = des_s16(d); // stored as u16, but actually is s16 - event.y = des_s16(d); // stored as u16, but actually is s16 - break; - } - - case EVENT.UNDO: - case EVENT.REDO: { - break; - } - - case EVENT.ERASER: { - event.stroke_id = des_u32(d); - break; - } - - default: { - console.error('fuck'); - } - } - - return event; -} - -function bitmap_bbox(event) { - const x = (event.x <= storage.canvas.width ? event.x : event.x - 65536); - const y = (event.y <= storage.canvas.height ? event.y : event.y - 65536); - - const bbox = { - 'xmin': x, - 'xmax': x + event.bitmap.width, - 'ymin': y, - 'ymax': y + event.bitmap.height - }; - - return bbox; -} - -async function handle_event(event) { - if (config.debug_print) console.debug(`event type ${event.type} from user ${event.user_id}`); - - // TODO(@speed): do not handle locally predicted events - - switch (event.type) { - case EVENT.STROKE: { - if (event.user_id in storage.predraw || event.user_id === storage.me.id) { - storage.predraw[event.user_id] = []; - redraw_predraw(); - } - - draw_stroke(event); - - break; - } - - case EVENT.UNDO: { - for (let i = storage.events.length - 1; i >=0; --i) { - const other_event = storage.events[i]; - - // Users can only undo their own, undeleted (not already undone) events - if (other_event.user_id === event.user_id && !other_event.deleted) { - if (other_event.type === EVENT.STROKE) { - other_event.deleted = true; - const stats = stroke_stats(other_event.points, storage.cursor.width); - redraw_region(stats.bbox); - break; - } else if (other_event.type === EVENT.IMAGE) { - other_event.deleted = true; - const item = document.querySelector(`img[data-image-id="${other_event.image_id}"]`); - if (item) item.remove(); - break; - } else if (other_event.type === EVENT.ERASER) { - other_event.deleted = true; - const erased = find_stroke_backwards(other_event.stroke_id); - if (erased) { - erased.deleted = false; - const stats = stroke_stats(erased.points, storage.cursor.width); - redraw_region(stats.bbox); - } - break; - } else if (other_event.type === EVENT.IMAGE_MOVE) { - const item = document.querySelector(`img[data-image-id="${other_event.image_id}"]`); - - const ix = storage.images[other_event.image_id].x -= other_event.x; - const iy = storage.images[other_event.image_id].y -= other_event.y; - - item.style.transform = `translate(${ix}px, ${iy}px)`; - - break; - } - } - } - - break; - } - - case EVENT.IMAGE: { - const url = config.image_url + event.image_id; - const item = document.createElement('img'); - - item.classList.add('floating-image'); - item.style['z-index'] = storage.events.length; - item.setAttribute('data-image-id', event.image_id); - item.setAttribute('src', url); - item.style.transform = `translate(${event.x}px, ${event.y}px)`; - - elements.images.appendChild(item); - storage.images[event.image_id] = { - 'x': event.x, 'y': event.y - }; - // const r = await fetch(config.image_url + event.image_id); - // const blob = await r.blob(); - // const bitmap = await createImageBitmap(blob); - // event.bitmap = bitmap; - // const bbox = bitmap_bbox(event); - // storage.ctx0.drawImage(bitmap, bbox.xmin, bbox.ymin); - - break; - } - - case EVENT.IMAGE_MOVE: { - // Already moved due to local prediction - if (event.user_id !== storage.me.id) { - const image_id = event.image_id; - const item = document.querySelector(`.floating-image[data-image-id="${image_id}"]`); - - const ix = storage.images[event.image_id].x += event.x; - const iy = storage.images[event.image_id].y += event.y; - - if (item) { - item.style.transform = `translate(${ix}px, ${iy}px)`; - } - } - - break; - } - - case EVENT.ERASER: { - if (event.deleted) { - break; - } - - for (const other_event of storage.events) { - if (other_event.type === EVENT.STROKE && other_event.stroke_id === event.stroke_id) { - // Might already be deleted because of local prediction - if (!other_event.deleted) { - other_event.deleted = true; - const stats = stroke_stats(other_event.points, storage.cursor.width); - redraw_region(stats.bbox); - } - break; - } - } - break; - } - - default: { - console.error('fuck'); - } - } -} - -async function handle_message(d) { - const message_type = des_u8(d); - - if (config.debug_print) console.debug(message_type); - - switch (message_type) { - case MESSAGE.JOIN: - case MESSAGE.INIT: { - elements.canvas0.classList.add('white'); - - storage.me.id = des_u32(d); - storage.server_lsn = des_u32(d); - - if (storage.server_lsn > storage.lsn) { - // Server knows something that we don't - storage.lsn = storage.server_lsn; - } - - if (message_type === MESSAGE.JOIN) { - ls.setItem('sessionId', des_u32(d)); - if (config.debug_print) console.debug('join in'); - } else { - if (config.debug_print) console.debug('init in'); - } - - const event_count = des_u32(d); - - if (config.debug_print) console.debug(`${event_count} events in init`); - - storage.ctx0.clearRect(0, 0, storage.ctx0.canvas.width, storage.ctx0.canvas.height); - elements.images.innerHTML = ''; - storage.events.length = 0; - - for (let i = 0; i < event_count; ++i) { - const event = des_event(d); - await handle_event(event); - storage.events.push(event); - } - - elements.canvas0.classList.remove('white'); - - send_ack(event_count); - - sync_queue(); - - break; - } - - case MESSAGE.FIRE: { - const user_id = des_u32(d); - const predraw_event = des_event(d); - - predraw_user(user_id, predraw_event); - - break; - } - - case MESSAGE.ACK: { - const lsn = des_u32(d); - - if (config.debug_print) console.debug(`ack ${lsn} in`); - - if (lsn > storage.server_lsn) { - // ACKs may arrive out of order - storage.server_lsn = lsn; - } - - break; - } - - case MESSAGE.SYN: { - const sn = des_u32(d); - const count = des_u32(d); - - const we_expect = sn - storage.sn; - const first = count - we_expect; - - if (config.debug_print) console.debug(`syn ${sn} in`); - - for (let i = 0; i < count; ++i) { - const event = des_event(d); - if (i >= first) { - handle_event(event); - storage.events.push(event); - } - } - - storage.sn = sn; - - await send_ack(sn); - - break; - } - - default: { - console.error('fuck'); - return; - } - } -} diff --git a/client/webgl.html b/client/webgl.html deleted file mode 100644 index 5db4379..0000000 --- a/client/webgl.html +++ /dev/null @@ -1,110 +0,0 @@ - - - - - Desk - - - - - - - - - - - - - - - - -
-
-
-
-
-
- -
-
- - \ No newline at end of file diff --git a/client/webgl.js b/client/webgl.js index e54f31e..6a108ee 100644 --- a/client/webgl.js +++ b/client/webgl.js @@ -58,7 +58,7 @@ function draw(state, context) { const total_pos_size = context.static_positions_f32.byteLength + context.dynamic_positions_f32.byteLength; const total_color_size = context.static_colors_u8.byteLength + context.dynamic_colors_u8.byteLength; - const total_point_count = (context.static_positions.length + context.dynamic_positions.length) / 2; + const total_point_count = (context.static_positions.length + total_dynamic_positions(context)) / 2; gl.bindBuffer(gl.ARRAY_BUFFER, buffers['b_pos']); gl.vertexAttribPointer(locations['a_pos'], 2, gl.FLOAT, false, 0, 0); @@ -79,16 +79,76 @@ const config = { ws_url: 'ws://192.168.100.2/ws/', image_url: 'http://192.168.100.2/images/', sync_timeout: 1000, - ws_reconnect_timeout: 2000, + ws_reconnect_timeout: 10000, second_finger_timeout: 500, buffer_first_touchmoves: 5, - debug_print: false, + debug_print: true, min_zoom: 0.01, max_zoom: 100.0, }; +const EVENT = Object.freeze({ + PREDRAW: 10, + STROKE: 20, + RULER: 21, /* gets re-written with EVENT.STROKE before sending to server */ + UNDO: 30, + REDO: 31, + IMAGE: 40, + IMAGE_MOVE: 41, + ERASER: 50, +}); + +const MESSAGE = Object.freeze({ + INIT: 100, + SYN: 101, + ACK: 102, + FULL: 103, + FIRE: 104, + JOIN: 105, +}); + +function event_size(event) { + let size = 1 + 3; // type + padding + + switch (event.type) { + case EVENT.PREDRAW: { + size += 4 * 2; + break; + } + + case EVENT.STROKE: { + size += 4 + 2 + 2 + 4 + event.points.length * 4 * 2; // u32 stroke id + u16 (count) + u16 (width) + u32 (color + count * (f32, f32) points + break; + } + + case EVENT.UNDO: + case EVENT.REDO: { + break; + } + + case EVENT.IMAGE: + case EVENT.IMAGE_MOVE: { + size += 4 + 4 + 4; // file id + x + y + break; + } + + case EVENT.ERASER: { + size += 4; // stroke id + break; + } + + default: { + console.error('fuck'); + } + } + + return size; +} + function main() { const state = { + 'me': 333, + 'canvas': { 'offset': { 'x': 0, 'y': 0 }, 'zoom': 1.0, @@ -99,6 +159,13 @@ function main() { 'y': 0, }, + 'sn': 0, + 'lsn': 0, + 'server_lsn': 0, + + 'color': 0, + 'stroke_width': 8, + 'touch': { 'moves': 0, 'drawing': false, @@ -116,18 +183,21 @@ function main() { 'stroke_width': 8, - 'current_stroke': { - 'color': 0, - 'width': 8, - 'points': [], - }, + 'current_strokes': {}, - 'strokes': [], + 'queue': [], + 'events': [], 'tools': { 'active': null, 'active_element': null, }, + + 'timers': { + 'ws_reconnect': null, + }, + + 'players': {}, }; const context = { @@ -137,12 +207,14 @@ function main() { 'buffers': {}, 'locations': {}, 'textures': {}, - 'static_positions': [], - 'dynamic_positions': [], + + 'dynamic_positions': {}, + 'dynamic_colors': {}, + 'quad_positions': [], 'quad_texcoords': [], + 'static_positions': [], 'static_colors': [], - 'dynamic_colors': [], 'static_positions_f32': new Float32Array(0), 'dynamic_positions_f32': new Float32Array(0), 'static_colors_u8': new Uint8Array(0), @@ -152,9 +224,16 @@ function main() { 'bgcolor': {'r': 1.0, 'g': 1.0, 'b': 1.0}, }; + const url = new URL(window.location.href); + const parts = url.pathname.split('/'); + + state.desk_id = parts.length > 0 ? parts[parts.length - 1] : 0; + init_webgl(state, context); init_listeners(state, context); init_tools(state, context); + + ws_connect(state, context, true); window.requestAnimationFrame(() => draw(state, context)); } \ No newline at end of file diff --git a/client/webgl_geometry.js b/client/webgl_geometry.js index 71e27af..e8cb430 100644 --- a/client/webgl_geometry.js +++ b/client/webgl_geometry.js @@ -95,39 +95,91 @@ function push_stroke(state, stroke, positions, colors) { } function pop_stroke(state, context) { - if (state.strokes.length > 0) { - const popped = state.strokes.pop(); + console.error('undo') + // if (state.strokes.length > 0) { + // // TODO: this will not work once we have multiple players + // // because there can be others strokes after mine + // console.error('TODO: multiplayer undo'); + + // const popped = state.strokes.pop(); - context.static_positions.length -= popped.popcount; - context.static_colors.length -= popped.popcount / 2 * 3; + // context.static_positions.length -= popped.popcount; + // context.static_colors.length -= popped.popcount / 2 * 3; + + // context.static_positions_f32 = new Float32Array(context.static_positions); + // context.static_colors_u8 = new Uint8Array(context.static_colors); + // } +} +function add_static_stroke(state, context, stroke, relax = false) { + push_stroke(state, stroke, context.static_positions, context.static_colors); + + if (!relax) { context.static_positions_f32 = new Float32Array(context.static_positions); context.static_colors_u8 = new Uint8Array(context.static_colors); } } -function add_static_stroke(state, context, stroke) { - state.strokes.push(stroke); - push_stroke(state, stroke, context.static_positions, context.static_colors); +function recompute_static_data(context) { context.static_positions_f32 = new Float32Array(context.static_positions); context.static_colors_u8 = new Uint8Array(context.static_colors); } -function update_dynamic_stroke(state, context, point) { - state.current_stroke.points.push(point); - context.dynamic_positions.length = 0; // TODO: incremental - context.dynamic_colors.length = 0; - push_stroke(state, state.current_stroke, context.dynamic_positions, context.dynamic_colors); - context.dynamic_positions_f32 = new Float32Array(context.dynamic_positions); - context.dynamic_colors_u8 = new Uint8Array(context.dynamic_colors); +function total_dynamic_positions(context) { + let total_dynamic_length = 0; + + for (const player_id in context.dynamic_positions) { + total_dynamic_length += context.dynamic_positions[player_id].length; + } + + return total_dynamic_length; +} + +function recompute_dynamic_data(state, context) { + const total_dynamic_length = total_dynamic_positions(context); + + context.dynamic_positions_f32 = new Float32Array(total_dynamic_length); + context.dynamic_colors_u8 = new Uint8Array(total_dynamic_length / 2 * 3); + + let at = 0; + + for (const player_id in context.dynamic_positions) { + context.dynamic_positions_f32.set(context.dynamic_positions[player_id], at); + at += context.dynamic_positions[player_id].length; + } + + // TODO: preview stroke colors + context.dynamic_colors_u8.fill(0); +} + +function update_dynamic_stroke(state, context, player_id, point) { + if (!(player_id in state.current_strokes)) { + state.current_strokes[player_id] = { + 'points': [], + 'width': 8, // TODO + 'color': 0, // TODO + }; + + context.dynamic_positions[player_id] = []; + context.dynamic_colors[player_id] = []; + } + + // TODO: incremental + context.dynamic_positions[player_id].length = 0; + context.dynamic_colors[player_id].length = 0; + + state.current_strokes[player_id].points.push(point); + push_stroke(state, state.current_strokes[player_id], context.dynamic_positions[player_id], context.dynamic_colors[player_id]); + + recompute_dynamic_data(state, context); } -function clear_dynamic_stroke(state, context) { - state.current_stroke.points.length = 0; - context.dynamic_positions.length = 0; - context.dynamic_colors.length = 0; - context.dynamic_positions_f32 = new Float32Array(0); - context.dynamic_colors_u8 = new Uint8Array(0); +function clear_dynamic_stroke(state, context, player_id) { + if (player_id in state.current_strokes) { + state.current_strokes[player_id].points.length = 0; + context.dynamic_positions[player_id].length = 0; + recompute_dynamic_data(state, context); + } } function add_image(context, bitmap, p) { diff --git a/client/webgl_listeners.js b/client/webgl_listeners.js index 4f55937..b975b9e 100644 --- a/client/webgl_listeners.js +++ b/client/webgl_listeners.js @@ -1,7 +1,7 @@ function init_listeners(state, context) { window.addEventListener('keydown', (e) => keydown(e, state, context)); window.addEventListener('keyup', (e) => keyup(e, state, context)); - + context.canvas.addEventListener('mousedown', (e) => mousedown(e, state, context)); context.canvas.addEventListener('mousemove', (e) => mousemove(e, state, context)); context.canvas.addEventListener('mouseup', (e) => mouseup(e, state, context)); @@ -53,8 +53,8 @@ function mousedown(e, state, context) { const screenp = {'x': e.clientX, 'y': e.clientY}; const canvasp = screen_to_canvas(state, screenp); - clear_dynamic_stroke(state, context); - update_dynamic_stroke(state, context, canvasp); + clear_dynamic_stroke(state, context, state.me); + update_dynamic_stroke(state, context, state.me, canvasp); state.drawing = true; window.requestAnimationFrame(() => draw(state, context)); @@ -73,11 +73,12 @@ function mousemove(e, state, context) { const screenp = {'x': e.clientX, 'y': e.clientY}; const canvasp = screen_to_canvas(state, screenp); - + state.cursor = screenp; if (state.drawing) { - update_dynamic_stroke(state, context, canvasp); + update_dynamic_stroke(state, context, state.me, canvasp); + fire_event(predraw_event(canvasp.x, canvasp.y)); do_draw = true; } @@ -101,13 +102,16 @@ function mouseup(e, state, context) { if (state.drawing) { const stroke = { - 'color': Math.round(Math.random() * 4294967295), - 'width': 8, //Math.round((Math.random() * 20) + 4), - 'points': process_stroke(state.current_stroke.points) + 'color': state.color, + 'width': state.stroke_width, + 'points': process_stroke(state.current_strokes[state.me].points), + 'user_id': state.me, }; add_static_stroke(state, context, stroke); - clear_dynamic_stroke(state, context); + queue_event(state, stroke_event(state)); + clear_dynamic_stroke(state, context, state.me); + state.drawing = false; window.requestAnimationFrame(() => draw(state, context)); @@ -151,11 +155,11 @@ function touchstart(e, state) { // Ingore subsequent touches if we are already drawing return; } - + // First finger(s) down? if (state.touch.ids.length === 0) { if (e.changedTouches.length === 1) { - + // We give a bit of time to add a second finger state.touch.waiting_for_second_finger = true; state.touch.moves = 0; @@ -184,18 +188,22 @@ function touchstart(e, state) { state.touch.first_finger_position = screenp; } else if (touch.identifier === state.touch.ids[1]) { state.touch.second_finger_position = screenp; - } + } } } return; } -} +} function touchmove(e, state, context) { if (state.touch.ids.length === 1) { const touch = find_touch(e.changedTouches, state.touch.ids[0]); + if (!touch) { + return; + } + const screenp = {'x': window.devicePixelRatio * touch.clientX, 'y': window.devicePixelRatio * touch.clientY}; const canvasp = screen_to_canvas(state, screenp); @@ -219,20 +227,18 @@ function touchmove(e, state, context) { } else { // Handle buffered moves if (state.touch.buffered.length > 0) { - clear_dynamic_stroke(state, context); + clear_dynamic_stroke(state, context, state.me); for (const p of state.touch.buffered) { - update_dynamic_stroke(state, context, p); - // const predraw = predraw_event(p.x, p.y); - // fire_event(predraw); + update_dynamic_stroke(state, context, state.me, p); + fire_event(predraw_event(canvasp.x, canvasp.y)); } state.touch.buffered.length = 0; } - // const predraw = predraw_event(x, y); - // fire_event(predraw); - update_dynamic_stroke(state, context, canvasp); + update_dynamic_stroke(state, context, state.me, canvasp); + fire_event(predraw_event(canvasp.x, canvasp.y)); window.requestAnimationFrame(() => draw(state, context)); } @@ -255,14 +261,14 @@ function touchmove(e, state, context) { first_finger_position = screenp; } else if (touch.identifier === state.touch.ids[1]) { second_finger_position = screenp; - } + } } const old_finger_midpoint = mid_v2(state.touch.first_finger_position, state.touch.second_finger_position); const new_finger_midpoint = mid_v2(first_finger_position, second_finger_position); const new_finger_midpoint_canvas = mid_v2( - screen_to_canvas(state, first_finger_position), + screen_to_canvas(state, first_finger_position), screen_to_canvas(state, second_finger_position) ); @@ -308,13 +314,15 @@ function touchend(e, state, context) { // await queue_event(event); const stroke = { - 'color': Math.round(Math.random() * 4294967295), - 'width': 8, // Math.round((Math.random() * 20) + 4), - 'points': process_stroke(state.current_stroke.points) + 'color': state.color, + 'width': state.stroke_width, + 'points': process_stroke(state.current_strokes[state.me].points), + 'user_id': state.me, }; add_static_stroke(state, context, stroke); - clear_dynamic_stroke(state, context); + queue_event(state, stroke_event(state)); + clear_dynamic_stroke(state, context, state.me); state.touch.drawing = false; window.requestAnimationFrame(() => draw(state, context)) diff --git a/client/websocket.js b/client/websocket.js index e2c1095..8e3db71 100644 --- a/client/websocket.js +++ b/client/websocket.js @@ -6,24 +6,24 @@ // // Details best described here: https://github.com/kee-org/KeeFox/issues/189 -function ws_connect(first_connect = false) { - const session_id = ls.getItem('sessionId') || '0'; - const desk_id = storage.desk_id; +function ws_connect(state, context, first_connect = false) { + const session_id = localStorage.getItem('sessionId') || '0'; + const desk_id = state.desk_id; ws = new WebSocket(`${config.ws_url}?deskId=${desk_id}&sessionId=${session_id}`); - ws.addEventListener('open', on_open); - ws.addEventListener('message', on_message); - ws.addEventListener('error', on_error); - ws.addEventListener('close', on_close); + ws.addEventListener('open', () => on_open(state)); + ws.addEventListener('message', (e) => on_message(state, context, e)); + ws.addEventListener('error', () => on_error(state, context)); + ws.addEventListener('close', () => on_close(state, context)); } -function on_open() { - clearTimeout(storage.timers.ws_reconnect); +function on_open(state) { + clearTimeout(state.timers.ws_reconnect); if (config.debug_print) console.debug('open') } -async function on_message(event) { +async function on_message(state, context, event) { const data = event.data; let message_data = null; @@ -31,7 +31,7 @@ async function on_message(event) { message_data = await data.arrayBuffer(); const view = new DataView(message_data); const d = deserializer_create(message_data, view); - await handle_message(d); + await handle_message(state, context, d); } else { /* For all my Safari < 14 bros out there */ const reader = new FileReader(); @@ -39,19 +39,19 @@ async function on_message(event) { message_data = e.target.result; const view = new DataView(message_data); const d = deserializer_create(message_data, view); - await handle_message(d); + await handle_message(state, context, d); }; reader.readAsArrayBuffer(data); } } -function on_close() { +function on_close(state, context) { ws = null; if (config.debug_print) console.debug('close'); - storage.timers.ws_reconnect = setTimeout(ws_connect, config.ws_reconnect_timeout); + state.timers.ws_reconnect = setTimeout(() => ws_connect(state, context, false), config.ws_reconnect_timeout); } -function on_error() { - ws.close(); +function on_error(state, context) { + ws.close(state, context); } \ No newline at end of file diff --git a/server/deserializer.js b/server/deserializer.js index 94f3714..76461ce 100644 --- a/server/deserializer.js +++ b/server/deserializer.js @@ -26,40 +26,53 @@ export function u32(d) { return value; } -function u16array(d, count) { - const array = new Uint16Array(d.view.buffer, d.offset, count); - d.offset += count * 2; +export function f32(d) { + const value = d.view.getFloat32(d.offset, true); + d.offset += 4; + return value; +} + +function f32array(d, count) { + const array = new Float32Array(d.view.buffer, d.offset, count); + d.offset += count * 4; return array; } +export function align(d, to) { + while (d.offset % to !== 0) { + d.offset++; + } +} + export function event(d) { const event = {}; event.type = u8(d); - u8(d); // padding switch (event.type) { case EVENT.PREDRAW: { - event.x = u16(d); - event.y = u16(d); + event.x = f32(d); + event.y = f32(d); break; } case EVENT.STROKE: { + // point_count + width align to 4 bytes :D const point_count = u16(d); const width = u16(d); const color = u32(d); + align(d, 4); event.width = width; event.color = color; - event.points = u16array(d, point_count * 2); + event.points = f32array(d, point_count * 2); break; } case EVENT.IMAGE: case EVENT.IMAGE_MOVE: { event.image_id = u32(d); - event.x = u16(d); - event.y = u16(d); + event.x = f32(d); + event.y = f32(d); break; } @@ -74,7 +87,7 @@ export function event(d) { } default: { - console.error('fuck'); + console.error('fuck', event.type); console.trace(); process.exit(1); } diff --git a/server/recv.js b/server/recv.js index 63f982d..6c0805a 100644 --- a/server/recv.js +++ b/server/recv.js @@ -42,7 +42,6 @@ function handle_event(session, event) { } async function recv_syn(d, session) { - const padding = des.u8(d); const lsn = des.u32(d); const count = des.u32(d); diff --git a/server/send.js b/server/send.js index eb7ef8c..5b31fa5 100644 --- a/server/send.js +++ b/server/send.js @@ -11,7 +11,7 @@ function event_size(event) { switch (event.type) { case EVENT.PREDRAW: { - size += 2 * 2; + size += 4 * 2; break; } @@ -23,7 +23,7 @@ function event_size(event) { case EVENT.IMAGE: case EVENT.IMAGE_MOVE: { - size += 4 + 2 + 2; // file id + x + y + size += 4 + 4 + 4; // file id + x + y break; } diff --git a/server/serializer.js b/server/serializer.js index c59c51d..afee7c0 100644 --- a/server/serializer.js +++ b/server/serializer.js @@ -21,6 +21,11 @@ export function u16(s, value) { s.offset += 2; } +export function f32(s, value) { + s.view.setFloat32(s.offset, value, true); + s.offset += 4; +} + export function u32(s, value) { s.view.setUint32(s.offset, value, true); s.offset += 4; @@ -37,15 +42,15 @@ export function event(s, event) { switch (event.type) { case EVENT.PREDRAW: { - u16(s, event.x); - u16(s, event.y); + f32(s, event.x); + f32(s, event.y); break; } case EVENT.STROKE: { const points_bytes = event.points; u32(s, event.stroke_id); - u16(s, points_bytes.byteLength / 2 / 2); // each point is 2 u16s + u16(s, points_bytes.byteLength / 2 / 4); // each point is 2 * f32 u16(s, event.width); u32(s, event.color); bytes(s, points_bytes); @@ -55,8 +60,8 @@ export function event(s, event) { case EVENT.IMAGE: case EVENT.IMAGE_MOVE: { u32(s, event.image_id); - u16(s, event.x); - u16(s, event.y); + f32(s, event.x); + f32(s, event.y); break; }