Currently every member of the community can invite you to join their groups. Optionally, you may restrict group invites to your connections only.