Can SQS scale up to 1,000,000 queues for a single account?

I need a messaging service that allows me to create a channel for each user in order to facilitate real-time notifications. If I have somewhere between 100,000 and 1 million users, does it make sense to create an SQS queue for each of these users?

According to the SQS pricing documentation it would only cost $0.40 to create 1 million queues, but would I run into scaling problems?

Also, is there a way to set an expiration date on a queue? If a user deletes their account, then their queue no longer needs to exist.


Creating queues is not an issue here. Polling or even long polling the queue is going to be really expensive for you. In order to process real-time notifications, you need to poll every queue, 1M of them for lets say every 5 seconds.

Based on SQS Pricing, Price per 1 Million Requests after free tier is $0.00000040 per request.

That means you will be calling the ReceiveMessage API for about:

1000000 queues * 17280 (1 day in seconds / 5 seconds) = 17280000000 times.

Which is about $6912.00 per day for the worst case scenarios.

You need to architect the solution in a better way.


Rabbitmq standalone and cluster installation

  • Install rabbitMQ in the VM. Following are the installations steps.
    ·         Verify if the earlang package is installed
  • rpm -q erlang-solutions-1.0-1.nonarch.rpm
  • wget
  • sudo wget
  • sudo yum update NOTE : use command “yum –releasever=6.7 update” if you want a specific version.
  • su -c ‘yum list rabbitmq’   Or use
  • yum install rabbitmq-server
  • sudo rpm -Uvh
  • sudo /etc/init.d/rabbitmq-server start·
  • Uncomment the loopback line in security section of rabbitMq.config :  {loopback_users, []}ss
  • rabbitmq-plugins enable rabbitmq_management·
  • Configure port firewall rule should be in place to accept the tcp connection.
  • Use following command : lokkit –p <rabbitMQ port>:tcp , lokkit –p <rabbitMQ management port>:tcp·
  • Default guest/guest account should be disabled. Change the user and user permissions using following commands :
  • Note : password should be 16 characters , no special characters allowed and should be generated by keypass.
  • rabbitmqctl set_user_tags <username> administrator      rabbitmqctl change_password guest guest123
  • Disable the guest user by changing the password once the created user is tested.
  • rabbitmqctl add_user <username> <password>
  • Avoid use of RabbitMQ default port and configure to use our own choice. Edit the port in rabbitMq.config file. uncomment following line and edit the port : {tcp_listeners, [<rabbitMQ port>]} and {listener, [{port,    <rabbitMQ management port>}.
  • Install management console of rabbitmq using following command :
  • Copy  /usr/share/doc/rabbitmq-server/ rabbitmq.config.example in /etc/rabbitmq folder and rename it as rabbitmq.config. Edit the permissions for the file to: 666
  • sudo chkconfig rabbitmq-server on
  • sudo rpm –import
    for rabbitmq 3.6.*  ,require socat dependency:
    steps : sudo yum install epel-release
    sudo yum install socat
  • sudo yum install -y erlang-18.2-1.el6
  • sudo rpm -Uvh erlang-solutions-1.0-1.noarch.rpm
  • Install erlang package:
  • dowload the erlang package from web site:
  • Restart the rabbitmq server using commnad : sudo service rabbitmq_server restart.
  • Make the following changes on rabbitmq console:  Got to Admin > click on user and click on set permissions. Check the permissions of the user. It should be same as user guest.
  • Try to create new queue to check it is working fine.


Create RabbitMQ High Availability Cluster:

1) Stop RabbitMQ in Master and slave nodes. Ensure service is stopped properly.

/etc/init.d/rabbitmq-server stop

2) Copy the file below to all nodes from the master. This cookie file needs to be the same across all nodes.

$ sudo cat /var/lib/rabbitmq/.erlang.cookie

3) Make sure you start all nodes after copying the cookie file from the master.

Start RabbitMQ in master and all nodes.

$ /etc/init.d/rabbitmq-server start

4) Then run the following commands in all the nodes, except the master node:

$ rabbitmqctl stop_app$ rabbitmqctl reset$ rabbitmqctl start_app

5) Now, run the following commands in the master node:

$ rabbitmqctl stop_app$ rabbitmqctl reset

6) Do not start the app yet.

Open port 4369 and 25672: lokkit -p 4369:tcp -p 25672:tcp

Stop the iptables on both master and slaves.

The following command is executed to join the slaves to the cluster:

$ rabbitmqctl join_cluster rabbit@slave1 rabbit@slave2

Update slave1 and slave2 with the hostnames/IP address of the slave nodes. You can add as many slave nodes as needed in the cluster.

7) Start master app in master machine

$ rabbitmqctl start_app

8) Check the cluster status from any node in the cluster:

$ rabbitmqctl cluster_status

9) In rabbitmq management console check if you can login with previous user and have all the previous settings in place.

If not create users by following command:

rabbitmqctl add_user <username> <password>

give admin rights:

rabbitmqctl set_user_tags <username> administrator

rabbitmqctl add_vhost /

Give vhost rights by:

rabbitmqctl set_permissions -p / <username> “.*” “.*” “.*”

10) Create ha mirroring by:

rabbitmqctl set_policy ha-all “” ‘{“ha-mode”:”all”,”ha-sync-mode”:”automatic”}’This will mirror all queues.

11) Now start iptables. You will have created rabbitmq HA cluster.