[aerogear-dev] GCM Topics - split into two sender implementations?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[aerogear-dev] GCM Topics - split into two sender implementations?

Lukáš Fryč
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).


What do you think?


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev
Reply | Threaded
Open this post in threaded view
|

Re: [aerogear-dev] GCM Topics - split into two sender implementations?

Matthias Wessendorf


On Wed, Aug 12, 2015 at 3:22 PM, Lukáš Fryč <[hidden email]> wrote:
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics

I agree that we need a slip here - but does it (GCM) really work like that ? 
I was more thinking/guess that we have to do the math and keep the device nr. per topic - wondering now, summers, what actually happens if device 1.000.001 registers w/ the topic? Does GCM fail the device ?
 
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).

but generally, I think these are two completely different cases, therefore we need to different senders 


What do you think?


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev



--

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev
Reply | Threaded
Open this post in threaded view
|

Re: [aerogear-dev] GCM Topics - split into two sender implementations?

Summers Pittman
In reply to this post by Lukáš Fryč


On Wed, Aug 12, 2015 at 9:22 AM, Lukáš Fryč <[hidden email]> wrote:
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).


What do you think?
Since we discussed this yesterday I am of course +1 :-p 


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev


_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev
Reply | Threaded
Open this post in threaded view
|

Re: [aerogear-dev] GCM Topics - split into two sender implementations?

Summers Pittman
In reply to this post by Matthias Wessendorf


On Wed, Aug 12, 2015 at 9:31 AM, Matthias Wessendorf <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 3:22 PM, Lukáš Fryč <[hidden email]> wrote:
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics

I agree that we need a slip here - but does it (GCM) really work like that ? 
I was more thinking/guess that we have to do the math and keep the device nr. per topic - wondering now, summers, what actually happens if device 1.000.001 registers w/ the topic? Does GCM fail the device ?

As far as Google's docs say, the device will register with GCM but the topics will fail to send on the server.

Specifically when we send the post to Google we will receive a failure messages TOO_MANY_DEVICES.  See the GCM Sender in the PR for exact details.

 
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).

but generally, I think these are two completely different cases, therefore we need to different senders 


What do you think?


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev



--

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev


_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev
Reply | Threaded
Open this post in threaded view
|

Re: [aerogear-dev] GCM Topics - split into two sender implementations?

Matthias Wessendorf


On Wed, Aug 12, 2015 at 3:36 PM, Summers Pittman <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 9:31 AM, Matthias Wessendorf <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 3:22 PM, Lukáš Fryč <[hidden email]> wrote:
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics

I agree that we need a slip here - but does it (GCM) really work like that ? 
I was more thinking/guess that we have to do the math and keep the device nr. per topic - wondering now, summers, what actually happens if device 1.000.001 registers w/ the topic? Does GCM fail the device ?

As far as Google's docs say, the device will register with GCM but the topics will fail to send on the server.

Specifically when we send the post to Google we will receive a failure messages TOO_MANY_DEVICES.  See the GCM Sender in the PR for exact details.

ok - thanks, so in case of this error - nothing is delivered? or just the first 1M ? 


 

 
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).

but generally, I think these are two completely different cases, therefore we need to different senders 


What do you think?


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev



--

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev


_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev



--

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev
Reply | Threaded
Open this post in threaded view
|

Re: [aerogear-dev] GCM Topics - split into two sender implementations?

Summers Pittman


On Wed, Aug 12, 2015 at 9:41 AM, Matthias Wessendorf <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 3:36 PM, Summers Pittman <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 9:31 AM, Matthias Wessendorf <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 3:22 PM, Lukáš Fryč <[hidden email]> wrote:
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics

I agree that we need a slip here - but does it (GCM) really work like that ? 
I was more thinking/guess that we have to do the math and keep the device nr. per topic - wondering now, summers, what actually happens if device 1.000.001 registers w/ the topic? Does GCM fail the device ?

As far as Google's docs say, the device will register with GCM but the topics will fail to send on the server.

Specifically when we send the post to Google we will receive a failure messages TOO_MANY_DEVICES.  See the GCM Sender in the PR for exact details.

ok - thanks, so in case of this error - nothing is delivered? or just the first 1M ? 
That is undocumented, but I would assume nothing is delivered. 


 

 
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).

but generally, I think these are two completely different cases, therefore we need to different senders 


What do you think?


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev



--

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev


_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev


_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev
Reply | Threaded
Open this post in threaded view
|

Re: [aerogear-dev] GCM Topics - split into two sender implementations?

Lukas Fryc
In reply to this post by Matthias Wessendorf


On Wed, Aug 12, 2015 at 3:31 PM, Matthias Wessendorf <[hidden email]> wrote:


On Wed, Aug 12, 2015 at 3:22 PM, Lukáš Fryč <[hidden email]> wrote:
Hi guys,


me and summersp have discussed how JMS should be used to route messages.


Current implementation loads tokens and conditionally sends either message with registration IDs or topics.

There are two things yet to be solved:
  •  topics can be used up to 1 million registrations, otherwise you have to fall back to enumerating registration IDs
  • implementation with one sender (GCMPushNotificationSender) is suboptimal
    • the utilization of the topics can be increased by prioritizing topic message sending first (covering multiple, potentially thousands registrations)
    • followed by sending registration IDs out

That's why I suggested to split implementation to two JMS queues talking to two sender impls (e.g. GCMTopicSender and GCMRegistrationIdsSender).
  • first we send out topic based messages (for efficiency)
  • we collect those topics that fail and resend them for processing as registration IDs (fail over)
    • registration ids sending can start in parallel, but it can't end until we sent out all topics

I agree that we need a slip here - but does it (GCM) really work like that ? 
I was more thinking/guess that we have to do the math and keep the device nr. per topic

Yes, that's certainly possible in a case we are able to find registration IDs for all devices subscribed to a given topic.

Actually our current strategy for detecting end of sending session is based on token (device) counting.

 
- wondering now, summers, what actually happens if device 1.000.001 registers w/ the topic? Does GCM fail the device ?
 
Additionally we get an ability to configure these two message routes individually on the JMS level (better utilization, transact-ability, fail over).

but generally, I think these are two completely different cases, therefore we need to different senders 


What do you think?


Cheers,

~ Lukas

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev



--

_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev


_______________________________________________
aerogear-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/aerogear-dev