For now only the same data channel messages that were already handled
are taken into account, but at a later point the missing messages
("speaking" and "stoppedSpeaking") could be added too.
Note that the thread used to handle the data channel messages has
changed; the EventBus subscriber mode was "MAIN", but as the messages
were posted from a DataChannel observer, which run in a worker thread
rather than in the main thread, the subscriber was executed in the main
thread rather than in the same thread as the poster. Due to this the
actions performed by the handler now must be explicitly run in the main
thread.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
If the call is a voice only call there will be no received video tracks
(they would have been stopped when each connection is established), so
changing the enabled state has no effect (as the adapter only tries to
show the received video if it is available).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
If the display name is not known whether "Guest" or something else needs
to be shown is not a responsibility of the web socket, so now an empty
string is returned instead.
In practice this should not make any difference, though, as the display
name of users is always known as soon as the user joined, and if the
nick of a guest is not known the UI will set it to "Guest".
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
PeerConnectionWrappers should not be concerned with the nick of
participants. Moreover, the nick is included in offers and answers due
to legacy reasons and only when the internal signaling server is used.
Due to that the nick was moved out of PeerConnectionWrapper; although
the handling is now different the end result should be the same (there
might be some differences in very specific sequences of events, but in
any case all this is just a temporary step and any leftover issue should
be addressed once call participants and peer connections are split).
As the PeerConnectionWrapper does not keep track of the nick now the
nick changed event is always emitted when a nick changed data channel
message is received, even if the nick did not actually change.
Nevertheless, before it was anyway always emitted if it was for a user
and only when it was for a guest it was emitted only on real changes. In
any case this is not expected to cause any issue (other than some
unneeded view updates, but that will be addressed at a later point by
updating the views only when the model actually changed).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
If the nick is not known whether "Guest" or something else needs to be
shown is a responsability of the UI, so now the PeerConnectionWrapper
just returns an empty string and the UI shows the default guest nick if
needed.
Moreover, the nick stored in the PeerConnectionWrapper was not always
correct, as if no nick was received it was returned as "Guest" even
if the connection belonged to a user. Now "Guest" is used only for
actual guests.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The generic data channel message works fine for receiving, but it could
not be used for sending, because the serialization of the payload failed
(the generated JsonMapper did not call 'writeFieldName("payload")',
apparently because the payload was defined as "Any", so there was no
field name set when serializing the payload contents).
It is very likely that the nick data channel message, which has an
explicit payload type and was used only for sending but not for
receiving, was added back in the day just to work around that
limitation. However, due to how the JsonMappers are generated if several
properties with the same name are defined only the first one will be
parsed, and only those with a value will be serialized. This makes
possible to define first a generic payload property and then a payload
property with an explicit type to have a single data channel message
class that can be used both for sending and receiving.
As the nick data channel message is now no longer needed it was removed.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Note that the thread used to send the message does not change; the
EventBus subscriber mode was "BACKGROUND", but as the messages were
posted from a WebSocket handler (when requesting offers to the HPB) and
peer connection observers (when sending offers/answers and candidates,
both with and without HPB), which run in worker threads rather than in
the main thread, the subscriber was executed in the same thread as
the poster.
For legacy reasons, when the internal signaling server is used the
offers and answers are expected to also provide the nick of the local
participant. When the external signaling server is used the field can be
included, but it is just ignored and not sent to the other clients. As
the local participant nick is a value unrelated to the peer connection
and is only needed with one type of signaling server the messages are
adjusted as needed before being sent rather than handling this inside
the PeerConnectionWrapper.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Like done with SignalingMessageReceiver, an implementation specific to
each signaling server type (internal or external) is added.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
As the "requestoffer" message is just a signaling message the generic
method can be used instead.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
"requestoffer" messages are compatible with the generic messages, so for
simplicity the generic message is used now instead of having specific
classes just for it.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Messages sent to the internal signaling server need to be serialized
twice, first the signaling message and then the wrapper as a whole. Due
to this the NCMessageWrapper was not actually used.
For simplicity the manual serialization was kept rather than adding
something like "NCMessageWrapperToSend" where a serialized signaling
message could be set before serializing it.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
NCMessageWrapper is used only for messages sent and received by the
internal signaling server. However, it is unused by the external
signaling server, except for getting the NCSignalingMessage, which is
the common message for both signaling servers.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Note that the thread used to handle the participant list messages from
the external signaling server does not change; the EventBus subscriber
mode was "BACKGROUND", but as the message was posted from a WebSocket
handler, which runs in a worker thread rather than in the main thread,
the subscriber was executed in the same thread as the poster.
Also note that the removed "userId" remark was not fully accurate;
although some external signaling messages do actually use "userid" those
currently handled to process the users do not, they always use "userId"
(as documented in the SignalingMessageReceiver).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
For now only the same participant list messages that were already
handled are taken into account, but at a later point further messages,
like participants joining or leaving the conversation, could be added
too.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Note that the thread used to handle and notify messages from the
external signaling server does not change; the EventBus subscriber mode
was "BACKGROUND", but as the message was posted from a WebSocket
handler, which runs in a worker thread rather than in the main thread,
the subscriber was executed in the same thread as the poster.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
This will no longer log an error if the room type of the received
message is neither "video" nor "screen". However, that should never
happen, and it would be useful only while debugging, so it is fine to
lose that.
Note that the check is not added to SignalingMessageReceiver itself to
keep it as generic as possible (and due to the low value of adding it as
explained above). Nevertheless, if needed in the future it would be
possible to add a special listener that receives raw messages in order
to validate them and log the errors, if any.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Although "unshareScreen" is technically bound to a specific peer
connection it is instead treated as a general message on the call
participant.
Nevertheless, call participant messages will make possible (at a later
point) to listen to events like "raise hand" or "mute" (which, again,
could be technically bound to a specific peer connection, but at least
for now are treated as a general message on the call participant).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Unlike the WebRtcMessageListener, which is bound to a specific peer
connection, an OfferMessageListener listens to all offer messages, no
matter which peer connection they are bound to. This can be used, for
example, to create a new peer connection when a remote offer for which
there is no previous connection is received.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Eventually all signaling related code should be moved to a Signaling
class that abstracts the differences between the internal and external
signaling servers, including how messages are sent and listened to. In
the meantime a temporary SignalingMessageReceiver implementation is
added to CallActivity to be able to start using it.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
For now only WebRTC messages can be listened to, although it will be
extended with other kinds later.
This commit only introduces the base class, although it is not used yet
anywhere; a concrete implementation will be added in a following commit.
The test class is named "SignalingMessageReceiverWebRtcTest" rather than
just "SignalingMessageReceiverTest" to have smaller, more manageable
test classes for each listener kind rather than one large test class for
all of them.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Although the rest of the methods are no longer needed since the handling
of WebRTC messages was moved to PeerConnectionWrapper "setSessionId()"
was not needed even before that.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
"peerConnectionWrapper" needs to be defined to enter the if and execute
the switch, so just return before the switch if "peerConnectionWrapper"
is null.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The message type is set for all signaling messages. On the other hand,
the payload type is only set for offers and answers (and, if the message
was sent by the Android app, also for candidates). However, in all those
cases the payload type just duplicates the message type, so the message
type can be assigned directly rather than falling back to it if there is
no payload type.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When the HPB is used the signaling messages can be received even when
the local participant is not currently in the call (for example, when
starting the call timed out without other participant joining, or when
reconnecting due to the publisher connection failing). Therefore if the
local participant is not in the call it should not try to establish a
connection with the other participants and disconnect them instead.
Moreover, if the connection is tried to be established when not
in the call the HPB will prevent that, and the PeerConnectionWrapper
will stay in a limbo state waiting for an offer to be sent. If the local
participant then joins the call the PeerConnectionWrapper will already
exist for the other participants, so no new connections will be created,
but those previous connections will never be finally established.
Additionally, as the signaling messages can be received before the join
call response the participant list could be received while the call
state is "RECONNECTING" or "PUBLISHER_FAILED". In those cases, as long
as the local participant is already in the call, the participant list
should be processed as if the call state was already "JOINED" (otherwise
the connections were not established either).
For simplicity the participant list is now ignored only when the call
state is "LEAVING"; this means that the participant list would be also
processed in the "CONNECTION_TIMEOUT" state, but the signaling message
should not be received anyway in that case.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
If the call is hung up with a view shutdown (which finishes the
activity) there is no need to do any further processing on the
participant list.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When a call is joined the call flags of the local participant change, so
this causes a signaling message to be sent by the server. When the HPB
is used the signaling message is sent through a WebSocket, which is
already connected before joining the call. Therefore, in some cases the
signaling message can be received through the WebSocket even before the
response to the HTTP "joinCall" request.
If there are other participants in the call the call state is changed to
"IN_CONVERSATION" when the signaling message is processed. However, in
the case described above the call state was then set to "JOINED", which
automatically traverses to "CONNECTION_TIMEOUT" if no other call state
was set in 45 seconds. Due to all this the call was joined and the
connections with the other participants were established, but they were
not visible in the UI (although they could be heard) and after 45
seconds the call was left.
To prevent that now the call state is changed to "JOINED" if it was not
already changed to "IN_CONVERSATION" in the meantime.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
did not test why this can happen. autocompleteUser.id must have been null..
Exception java.lang.NullPointerException:
at com.nextcloud.talk.controllers.ContactsController.processAutocompleteUserList (ContactsController.kt:498)
at com.nextcloud.talk.controllers.ContactsController.processAutocompleteUserList (ContactsController.kt:482)
at com.nextcloud.talk.controllers.ContactsController.access$processAutocompleteUserList (ContactsController.kt:90)
at com.nextcloud.talk.controllers.ContactsController$fetchData$1.onNext (ContactsController.kt:438)
at com.nextcloud.talk.controllers.ContactsController$fetchData$1.onNext (ContactsController.kt:432)
at io.reactivex.internal.operators.observable.ObservableRetryPredicate$RepeatObserver.onNext (ObservableRetryPredicate.java:69)
at io.reactivex.internal.operators.observable.ObservableObserveOn$ObserveOnObserver.drainNormal (ObservableObserveOn.java:201)
at io.reactivex.internal.operators.observable.ObservableObserveOn$ObserveOnObserver.run (ObservableObserveOn.java:255)
at io.reactivex.android.schedulers.HandlerScheduler$ScheduledRunnable.run (HandlerScheduler.java:124)
at android.os.Handler.handleCallback (Handler.java:883)
at android.os.Handler.dispatchMessage (Handler.java:100)
at android.os.Looper.loop (Looper.java:237)
at android.app.ActivityThread.main (ActivityThread.java:7948)
at java.lang.reflect.Method.invoke
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:493)
at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1075)
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
The module class is not supposed to have things injected into it. @Provides-annotated methods
will have their parameters injected, instead.
Signed-off-by: Álvaro Brey <alvaro.brey@nextcloud.com>
Possible null pointer dereference in com.nextcloud.talk.utils.PushUtils.updatePushStateForUser(Map, User) due to return value of called method
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
PRMC: In class com.nextcloud.talk.utils.ssl.MagicKeyManager
In class com.nextcloud.talk.utils.ssl.MagicKeyManager
In method com.nextcloud.talk.utils.ssl.MagicKeyManager.chooseClientAlias(String[], Principal[], Socket)
At MagicKeyManager.java:[line 68]
Value getCurrentUser()Lio/reactivex/Maybe;
Method com.nextcloud.talk.utils.ssl.MagicKeyManager.chooseClientAlias(String[], Principal[], Socket) appears to call the same method on the same object redundantly
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
This is to work around a race condition where this class would asynchronously overwrite other user attributes
with old values after a user switch.
Co-authored-by: Marcel Hibbe <dev@mhibbe.de>
Signed-off-by: Álvaro Brey <alvaro.brey@nextcloud.com>
i was able to see that the loop was somehow interrupted during debugging which caused two users to have current =true
this should avoid the problem with the loop.
anyway, this doesn't seem to solve the issue completely as i was able to reproduce it again with the new solution. so maybe there are still more methods/scenarios which can cause this.
additionally, i managed to have all users to have current =false with this new query (while switching accounts very fast and often in ChooseAccountDialogFragment..)
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
this should have no effect but it should ensure "current" is not falsely set to true if the method storeProfile will be used for more scenarios in the future.
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
just refactoring for now. this doesn't solve the bug!
Problem that needs to be solved:
When adding a new Account (User), it is marked as "current", while for the other logged in users "current" must be unset (-> disabled).
The problem is, that for the old active user, "current" is not unset so there were multiple accounts marked as "current".
In the ChooseAccountDialogFragment, only one of the current accounts is shown at the top. Below the set status field, all accounts are listed that are not marked with "current". So as a result, there can be accounts hidden that were marked as "current".
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
-> add callback methods to ConductorRemapping to execute after chat room was left. Whenever there is a ChatController on top, it's room is now left, before replacing the controller or pushing another one on top.
this avoids problems where entering a chat before the old one was left led to sessionId="0" for the new chat.
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
when opening a chat notification, the old chat should not be kept in the backstack. so when clicking the back button when coming from a chat that was opened by a notification, now the ConversationList opens.
by the way, this also avoids to run into bug #2181 (but it's root cause is not solved yet)
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
# reproduce:
1. click in chat where someone is mentioned.
2. click on the mention
3. go back to first chat
# result:
validSessionId() can be false in onDetach for the second chat when going back to first chat
-> leaveRoom is not executed
-> disposable is not disposed
-> getRoomInfo() continues to execute for old controller
-> e.g. appbar infos can be wrong (wrong avatar/title)
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
set drawable of referenceThumbImage to null. this hopefully avoids that wrong images are loaded from recycler view to wrong messages
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
there was a report that a message contained the link preview of a previous message. This was most likely because of a recycler view error. Hopefully setting empty values should avoid this now.
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
for example when call is hangup on mobile and immediately after on web, the loop in "checkIfCallIsActive" is still active and might trigger to send the "missed call" notification. Because of this, there is now another check if the "ongoing call" notification is still visible. It makes only sense to show the missed call notification, when the ongoing call notification is still visible.
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
this commit removes the logic to play the ringtone in CallNotificationActivity. Playing ringtone should only be controlled by the notification channel from OS!
furthermore the checks if a call is stopped or is still ongoing etc was removed from CallNotificationActivity. Instead the CallNotificationActivity now is completely dependent on the notification. If the notification is canceled, the Activity stops. If the Notification is ongoing and hangup of accept call is clicked, then the notification is canceled (including the ringtone).
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
Fresco is replaced with Coil everywhere to make it possible to set 'minSdkVersion'
to 23. But Coil is not used directly to avoid splintering the dependency
everywhere in the code. Coil is wrapped by extension functions for 'ImageView'.
Some shared functionality is moved from 'DisplayUtils' into the
'ImageViewExtensions'.
The exisiting initialization of Coil has also be changed. The usage of the self
initialized OKHttp client is removed. If this one is added the
caching of the http client is used by Coil additionally to memory and
disk cache.
Resolves: #2227, #2376
Signed-off-by: Tim Krüger <t@timkrueger.me>
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.nextcloud.talk2, PID: 10874
java.lang.NullPointerException: uri param can not be null.
at android.media.MediaPlayer.setDataSource(MediaPlayer.java:1058)
at android.media.MediaPlayer.setDataSource(MediaPlayer.java:1021)
at com.nextcloud.talk.activities.CallActivity.playCallingSound(CallActivity.java:2643)
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
When the web socket is abruptly closed it is connected again and the
call is rejoined. However, the call was rejoined in a background thread,
so an exception was thrown when trying to modify the views, which
prevented the call from being joined again.
Besides that the call state needs to be explicitly changed, as if the
web socket was connected again while in a call the state would be
already "JOINED" or "IN_CONVERSATION", which prevents the signaling
settings from being fetched again after the permissions check, and
therefore also prevented the call from being joined again.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The call flags should be checked using bitwise operators; otherwise a
call with phone ("in call | with phone" / "in call + with phone", that
is, "0001 + 1000 = 1001" or "1 + 8 = 9") would be seen as a call with
video ("in call | with video" / "in call + with video",
"0001 + 0100 = 0101" or "1 + 4 = 5"), as "9 >= 5". On the other hand,
using bitwise operators (and only checking against "with video")
succeeds only when the call flags contain "with video" (in the previous
example, "1001 and 0100 = 0000", so it does not succeed).
The "IN_CALL" flag is no longer checked, as "WITH_VIDEO" will be set
only during calls, and therefore checking for "IN_CALL" is not needed.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
In further versions of the ImagePicker library the class 'File' is used
to reference an image. Using 'File' caused the permission problems mentioned
in #2511.
Resolves: #2511
See: d7e643b560
Signed-off-by: Tim Krüger <t@timkrueger.me>
The ParticipantDisplayItems were associated to both the session ID and
the video stream type ("video" or "screen"), but the code that gets them
was not updated to include the video stream type in the key.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
ParticipantDisplayItems are not associated to a full participant but to
each of the single connections that the participant may have (video and
screen). However, when they are added to the map only the session ID is
used as key. Due to this when a participant starts a screen share the
ParticipantDisplayItem for the screen share overwrites the item for the
video, and once the screen share is stopped the old item is not
restored. Moreover, if a participant is already sharing a screen when
the local participant joins whether the video or the screen share is
shown is undefined and depends on which connection is established first.
To solve that the ParticipantDisplayItems are now associated with both
the session ID and the video stream type ("video" or "screen"). Due to
this both the video and the screen share of the remote participant are
shown in the grid view; in the future it might be better to only show
the screen share, or allow switching between screen share and video, or
show the screen share in full screen and hide the grid... but for now,
as a quick fix, this is good enough :-)
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
this didn't make sense because time between firebase and devices is not synchronized, so the results were useless.
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
