Commit graph

53 commits

Author SHA1 Message Date
Travis Ralston
55eddae096
Merge pull request #6035 from matrix-org/travis/voicemessages/playback-time
Improve progress bar progression for smaller voice messages
2021-05-14 10:17:35 -06:00
Travis Ralston
ddf9027daa Improve progress bar progression for smaller voice messages
Instead of chunking the thing, we'll improve our precision and clock accuracy. Clock accuracy is improved by flagging the "load time" of the clip in the context, which can be about 500ms (or more) off the context's start line. The precision is just a number in the PlaybackWaveform component.
2021-05-13 22:35:43 -06:00
Travis Ralston
df7b598ca7 Improve visible waveform for voice messages
This tries to prioritize actual voice to decide the waveform, and clamps noise to zero to ensure the waveform doesn't have a perceptually noisy base.

In theory this better matches the overall voice message content.
2021-05-13 22:20:08 -06:00
Travis Ralston
b1cb2b1d93 Fix bad merge 2021-05-11 10:19:09 -06:00
Travis Ralston
d8e3458c55 Merge remote-tracking branch 'origin/develop' into travis/voicemessages/resample2 2021-05-11 10:15:37 -06:00
Travis Ralston
b007ea81b2 Rescale and smooth playback waveform to better match expectation 2021-05-07 21:06:07 -06:00
Travis Ralston
65f591b69b Make the tests happier
Here we just override the workers because we're not expecting to be able to test them this way. The code paths involved shouldn't be touched.
2021-05-06 22:08:00 -06:00
Travis Ralston
f65773ef95 Appease the linter 2021-05-06 21:49:53 -06:00
Travis Ralston
b08e47bfe1 Support compatibility points for Safari
Tested on MacOS Big Sur, Safari 14.0.3
2021-05-06 21:43:17 -06:00
Travis Ralston
b61fe2f8e6 Improve error recovery when starting a recording
This helps return the microphone access to the user.
2021-05-06 21:43:17 -06:00
Travis Ralston
ed43d92579 Match designs more closely with +1 sample 2021-05-04 20:55:35 -06:00
Travis Ralston
a94c1a90c1 Update colours and sizing for voice messages
Fixes https://github.com/vector-im/element-web/issues/17162
2021-05-04 20:45:15 -06:00
Travis Ralston
54931cb43e Merge remote-tracking branch 'origin/develop' into travis/voicemessages/timeline 2021-05-04 09:05:04 -06:00
Travis Ralston
7f56ec7bd2 docs 2021-05-03 15:16:50 -06:00
Travis Ralston
0806af0f4d Calculate the real waveform in the Playback class 2021-05-03 15:07:52 -06:00
Travis Ralston
3f4ee9742a Simple rendering of voice messages in the timeline
Fixes https://github.com/vector-im/element-web/issues/17138
2021-05-03 11:41:14 -06:00
Travis Ralston
f0ff2fc38d Ensure we capture an absolute maximum amount of audio samples
We say the limit is 2 minutes, not 1m59s, so let's give the user that last frame.
2021-04-27 23:30:54 -06:00
Travis Ralston
c2bcdae8a9 Switch global var to the store for easier debugging 2021-04-27 23:04:49 -06:00
Travis Ralston
c2d37af1cb Move playback to its own set of classes
This all started with a bug where the clock wouldn't update appropriately, and ended with a whole refactoring to support later playback in the timeline.

Playback and recording instances are now independent, and this applies to the <Playback* /> components as well. Instead of those playback components taking a recording, they take a playback instance which has all the information the components need.

The clock was incredibly difficult to do because of the audio context's time tracking and the source's inability to say where it is at in the buffer/in time. This means we have to track when we started playing the clip so we can capture the audio context's current time, which may be a few seconds by the first time the user hits play. We also track stops so we know when to reset that flag.

Waveform calculations have also been moved into the base component, deduplicating the math a bit.
2021-04-27 22:23:57 -06:00
Travis Ralston
5e646f861c Wire up the send button for voice messages
This fixes a bug where we couldn't upload voice messages because the audio buffer was being read, therefore changing the position of the cursor. When this happened, the upload function would claim that the buffer was empty and could not be read.
2021-04-27 18:59:10 -06:00
Travis Ralston
30e120284d Add simple play/pause controls 2021-04-27 15:02:15 -06:00
Travis Ralston
32e3ce3dea Handle basic state machine of recordings 2021-04-27 15:02:15 -06:00
Travis Ralston
14809dfda7 Misc cleanup 2021-04-22 08:22:31 -06:00
Travis Ralston
b519d85127
Update src/voice/RecorderWorklet.ts to use sanity
Co-authored-by: Germain <germain@souquet.com>
2021-04-20 09:32:12 -06:00
Travis Ralston
4f75e2944c Appease the linter 2021-04-19 23:11:41 -06:00
Travis Ralston
61730f2f88 Populate waveform data on voice message event 2021-04-19 23:05:53 -06:00
Travis Ralston
7d9562137e Replace deprecated processor with a worklet 2021-04-19 21:54:08 -06:00
Travis Ralston
6f794cca9b Fill in some metadata for the sent event 2021-04-19 21:01:48 -06:00
Travis Ralston
22233a8745 Add a concept of a singleflight to avoid repeated calls to stop/ending
This makes it easier to keep track of which pieces the client will have already dispatched or been executed, reducing the amount of class members needed.

Critically, this makes it so the 'stop' button (which is currently a send button) actually works even after the automatic stop has happened.

UI is still pending for stopping recording early. This is not covered by this change.
2021-04-14 21:59:50 -06:00
Travis Ralston
0677cf866c Cap recording length, and warn at 10s remaining
See diff for details. Note that this introduces an "Uploading" state which is not currently used.

At the moment, if a user hits the maximum time then their recording will be broken. This is expected to be fixed in a future PR.
2021-04-14 21:15:06 -06:00
Travis Ralston
b0a04c9f81 Rename VoiceRecorder -> VoiceRecording to better match expected function 2021-04-14 20:00:16 -06:00
Travis Ralston
b15412056e It helps to use the right function 2021-03-30 12:08:11 -06:00
Travis Ralston
60326e359a Clarify comment 2021-03-30 12:01:35 -06:00
Travis Ralston
54412878a1 Merge branch 'develop' into travis/voice-messages/waveform 2021-03-29 22:59:51 -06:00
Travis Ralston
9c2d44805d Revert "Avoid use of deprecated APIs, instead using an AudioWorklet"
This reverts commit 5c685dcf35.
2021-03-29 22:57:59 -06:00
Travis Ralston
5c685dcf35 Avoid use of deprecated APIs, instead using an AudioWorklet
A bit annoying that it is async, but it'll do.
2021-03-29 21:59:31 -06:00
Travis Ralston
e523ce6036 Fix float operations to make a little more sense. 2021-03-29 21:25:06 -06:00
Travis Ralston
e31ad0e174 Don't specify sample rates for voice messages
Turns out the browser doesn't actually resample for us, instead opting to explode in sadness.

We'll leave the resampling to the opus encoder.

Fixes https://github.com/vector-im/element-web/issues/16775
2021-03-25 18:24:58 -06:00
Travis Ralston
101679f647 Adjust some settings 2021-03-25 17:30:44 -06:00
Travis Ralston
1419ac6b69 Hook up a clock and implement proper design 2021-03-25 17:12:26 -06:00
Travis Ralston
449e028bbd Actually use a waveform instead of the frequency data 2021-03-24 23:31:02 -06:00
Travis Ralston
8ddd14e252 Early concept for rendering the frequency waveform 2021-03-24 22:17:45 -06:00
Travis Ralston
d929d48391 Clean up promises 2021-03-23 18:26:43 -06:00
Travis Ralston
c9938ff704 Adjust settings/docs for encoder 2021-03-23 18:24:40 -06:00
Travis Ralston
090cf28af4 Appease the linter 2021-03-22 19:36:58 -06:00
Travis Ralston
e352ed1908 Run audio through the Web Audio API instead
This leads to more reliable frequency/timing information, and involves a whole lot less decoding.

We still maintain ongoing encoded frames to avoid having to do one giant encode at the end, as that could take long enough to be disruptive.
2021-03-22 19:32:37 -06:00
Travis Ralston
207ba11da1 Tweak a bunch of settings 2021-03-19 17:08:01 -06:00
Travis Ralston
6c566cd382 Remove debugging 2021-03-16 23:50:03 -06:00
Travis Ralston
1dc2427128 Remove test function 2021-03-16 23:44:39 -06:00
Travis Ralston
9aa5348c7f Show a stop button to stop the recording 2021-03-16 23:43:59 -06:00