Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inaccessible host: localhost'. This service may not be available in the us-east-1' region. #125

Closed
ravithejappl opened this issue Dec 28, 2020 · 43 comments

Comments

@ravithejappl
Copy link

Can't I run this repo without aws account?

@whummer
Copy link
Member

whummer commented Dec 28, 2020

Thanks for reporting @ravithejappl . Yes, you should be able to use the serverless-localstack plugin without an AWS account. Can you please share the details of your environment (operating system, Python/Docker version, etc.) and project setup (serverless.yml), to help us reproduce this issue? Thanks

@ravithejappl
Copy link
Author

ravithejappl commented Dec 30, 2020

I am using macos catalina(10.15.5), docker engine 20.10.0,
I am running this repo example only

I am getting below error
Inaccessible host: localhost'. This service may not be available in the us-east-1' region.
@whummer can you reply me as early as possible? We are dependent on it.

@shreekant89
Copy link

i am also getting same error i am using nodejs,docker latest,and localstock latest version
and error is "Inaccessible host: localhost'. This service may not be available in the us-east-1' region."

@amouly
Copy link

amouly commented Mar 15, 2021

Same issue here:

Inaccessible host: 'localhost'. This service may not be available in the 'ap-southeast-1' region.

@NimaSoroush
Copy link

Same here. I'm using amplifylocal init and getting this on an existing project

@juanstiza
Copy link

Same issue here when trying to send messages to an SQS queue.

@liam-ot
Copy link

liam-ot commented Sep 23, 2021

if using docker check out this link

@edbond
Copy link

edbond commented Nov 22, 2021

Same here, started localstack start and sls deploy --stage local gives this:

Serverless: Recoverable error occurred (Inaccessible host: `localhost' at port `4566'. This service may not be available in the `us-west-1' region.), sleeping for ~7 seconds. Try 1 of 4

localstack says it's listen on 4566

Starting edge router (https port 4566)...
Ready.
[2021-11-22 19:48:34 +0000] [23] [INFO] Running on https://0.0.0.0:4566 (CTRL + C to quit)
2021-11-22T19:48:34.469:INFO:hypercorn.error: Running on https://0.0.0.0:4566 (CTRL + C to quit)

How is this supposed to work?

@edbond
Copy link

edbond commented Nov 22, 2021

Fixed by running docker to start localstack
docker run --rm -it -p 4566:4566 -p 4571:4571 localstack/localstack

instead of localstack start

@elangoram1998
Copy link

I am getting the same issue when I try to invoke SQS queue from a lambda function. I tried the same step which @edbond mentioned above, but that didn't seems to be working.

local stack version: 0.14.1.3
docker engine: 20.10.13

In my lambda:
new AWS.SQS({
endpoint: 'http://localhost:4566',
region: 'us-east-1',
});

Actually s3 is working by using below, but not sqs.
new AWS.S3({
endpoint: 'http://localhost:4566',
region: 'us-east-1',
s3ForcePathStyle: true,
});

Note: I am able to send sqs message from aws cli, but not from local stack lambda function.

@omerls-pw
Copy link

This error still happening.

@fisher-he
Copy link

I found that this error happen with serverless 2.x. If I upgrade the serverless to 3.x, this error will be resolved.
But I still doubt why serverless-localstack plugin does not support serverless 2.x.

@omerls-pw
Copy link

I found that this error happen with serverless 2.x. If I upgrade the serverless to 3.x, this error will be resolved. But I still doubt why serverless-localstack plugin does not support serverless 2.x.

I'm using serverless 3.x

$ sls --version
Framework Core: 3.26.0
Plugin: 6.2.2
SDK: 4.3.2

@homeyer
Copy link

homeyer commented Jan 12, 2023

The default serverless host is using localhost, which on a mac is going to resolve to the IPv6 version instead of IPv4 127.0.0.1. Serverless is expecting the IPv4 address.

In your serverless config, override the host with host: 127.0.0.1 to force the resolution.

@ze-ahmed
Copy link

ze-ahmed commented Jan 19, 2023

I'm still experiencing the same issue using Serverless 3

Error:
Inaccessible host: `localhost' at port `4566'. This service may not be available in the `us-east-1' region.

With @homeyer solution I got:

Error: Hostname/IP does not match certificate's altnames: IP: 127.0.0.1 is not in the cert's list:

@aron-airgraft
Copy link

I'm still experiencing the same issue using Serverless 3

Error:
Inaccessible host: `localhost' at port `4566'. This service may not be available in the `us-east-1' region.

With @homeyer solution I got:

Error: Hostname/IP does not match certificate's altnames: IP: 127.0.0.1 is not in the cert's list:

I was able to circumvent it with host: http://127.0.0.1 in the localstack custom config.

@ze-ahmed
Copy link

ze-ahmed commented Jan 19, 2023

@aron-airgraft tried it also and got the following error

Skipping template validation: Unsupported in Localstack

✖ Stack app-002-local failed to deploy (19s)
Environment: darwin, node 19.0.0, framework 3.26.0, plugin 6.2.2, SDK 4.3.2
Credentials: Local, "default" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
CREATE_FAILED: app-002-local (AWS::CloudFormation::Stack)
undefined

View the full error: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stack/detail?stackId=arn%3Aaws%3Acloudformation%3Aus-east-1%3A000000000000%3Astack%2Fapp-002-local%2F4563fr1

@whummer
Copy link
Member

whummer commented Jan 20, 2023

The default serverless host is using localhost, which on a mac is going to resolve to the IPv6 version instead of IPv4 127.0.0.1. Serverless is expecting the IPv4 address.
In your serverless config, override the host with host: 127.0.0.1 to force the resolution.

Great catch, thanks for sharing the fix/workaround @homeyer !

Can you confirm if this entry in the config resolves the issue for you @ze-ahmed @aron-airgraft @omerls-pw @fisher-he @elangoram1998 @edbond @ravithejappl @amouly ?

custom:
  localstack:
    ...
    host: http://127.0.0.1

We'll look into getting this fix into the plugin itself, to make the switch between IPv4/IPv6 endpoints automatically. 👍

@ze-ahmed
Copy link

Thanks a lot for your help.
For me it worked with @homeyer solution + by adding HOSTNAME=127.0.0.1 in the localstack env profile

@aron-airgraft
Copy link

Can you confirm if this entry in the config resolves the issue for you

Gets most of the way there but I ran into Hostname/IP does not match certificate's altnames: IP: 127.0.0.1 is not in the cert's list when using just host: 127.0.0.1, which I got around with host: http://127.0.0.1.

@whummer
Copy link
Member

whummer commented Jan 20, 2023

Thanks for confirming @ze-ahmed @aron-airgraft . We have now pushed a few enhancements in version [email protected] - can you please give it another try with this version (and without using the custom host override). Thanks!

@ze-ahmed
Copy link

@whummer thanks a lot, it works well for me I've removed the host property in serverless.yml and also the HOSTNAME=127.0.0.1 from the localstack env file

@joe4dev
Copy link
Member

joe4dev commented Feb 1, 2023

@whummer the problem still requires workarounds for newer Nodejs versions because v17 changed a default behavior such that IPv6 is favored over IPv4. This breaks the defaults for LocalStack running on IPv4.
See localstack/aws-cdk-local#76 (comment)

Workarounds:

  • host: http://127.0.0.1 in serverless-localstack
  • export LOCALSTACK_HOSTNAME=127.0.0.1 in cdklocal

@whummer
Copy link
Member

whummer commented Feb 10, 2023

Thanks for the update @joe4dev . 👍 Do you think we could somehow extend the check in getConnectHostname to also transparently resolve the issues with Node.js v17?

@joe4dev
Copy link
Member

joe4dev commented Feb 14, 2023

@Kalanamith
Copy link

This error exists in Macbook M2 sillicon ship machines

@joe4dev
Copy link
Member

joe4dev commented Feb 16, 2023

@Kalanamith We just (2023-02-15T19:17:18.227Z) released serverless-localstack 1.0.4, which implements an IPv4 fallback to fix IPv6-related issues on macOS. Can you please check with the latest version?

developersteve added a commit to developersteve/localstack-demo that referenced this issue Feb 20, 2023
Updating localstack package version to latest to fix issues around host (see localstack/serverless-localstack#125)
@ramsey
Copy link

ramsey commented Feb 25, 2023

I'm also experiencing this problem on an M1 Max, and I'm using serverless-localstack version 1.0.4.

My package.json looks like this:

{
  "devDependencies": {
    "serverless": "^3.27.0",
    "serverless-localstack": "^1.0.4"
  },
  "resolutions": {
      "**/aws-sdk": "2.1313.0"
  }
}

serverless.yaml contains:

service: bref-test

provider:
  name: aws
  region: us-east-1
  runtime: provided.al2
  layers:
    - ${bref:layer.php-82}
  deploymentMethod: direct

plugins:
  - ./vendor/bref/bref
  - serverless-localstack

custom:
  localstack:
    stages:
      - local
    host: http://127.0.0.1
    debug: true

functions:
  hello:
    handler: index.php

I'm attempting to deploy with the following command:

AWS_PROFILE=localstack yarn serverless deploy --stage local

And this is the output I see:

[...]

Deploying bref-test to stage local (us-east-1)
config.options_stage: local
serverless.service.custom.stage: undefined
serverless.service.provider.stage: dev
config.stage: local
config.options_stage: local
serverless.service.custom.stage: undefined
serverless.service.provider.stage: dev
config.stage: local
Using custom endpoint for STS: http://127.0.0.1:4566
Using custom endpoint for CloudFormation: http://127.0.0.1:4566
Using custom endpoint for CloudFormation: http://127.0.0.1:4566
Using custom endpoint for CloudFormation: http://127.0.0.1:4566
Using custom endpoint for CloudFormation: http://127.0.0.1:4566
Using custom endpoint for S3: http://127.0.0.1:4566
Using custom endpoint for S3: http://127.0.0.1:4566
Using custom endpoint for Lambda: http://127.0.0.1:4566
Using custom endpoint for S3: http://127.0.0.1:4566
Using custom endpoint for S3: http://127.0.0.1:4566
Using custom endpoint for S3: http://127.0.0.1:4566
Skipping template validation: Unsupported in Localstack
Using custom endpoint for CloudFormation: http://127.0.0.1:4566
Overriding S3 templateUrl to http://127.0.0.1:4566
Using custom endpoint for CloudFormation: http://127.0.0.1:4566

✖ Stack bref-test-local failed to deploy (11s)
Environment: darwin, node 16.13.1, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2
Credentials: Local, "localstack" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
UPDATE_FAILED: bref-test-local (AWS::CloudFormation::Stack)
undefined

View the full error: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stack/detail?stackId=arn%3Aaws%3Acloudformation%3Aus-east-1%3A000000000000%3Astack%2Fbref-test-local%2Fa5ced7cf
error Command failed with exit code 1.

@ramsey
Copy link

ramsey commented Feb 25, 2023

After fiddling around with this some more, I tried @edbond's tip and started localstack directly from docker instead of using localstack start:

docker run --rm -it -p 4566:4566 localstack/localstack

Now, when I run the following, it appears to start:

AWS_PROFILE=localstack yarn serverless deploy --stage local
[...]

✔ Service deployed to stack bref-test-local (13s)

functions:
  hello: bref-test-local-hello (1.1 MB)

Need a better logging experience than CloudWatch? Try our Dev Mode in console: run "serverless --console"
✨  Done in 14.42s.

However, when I try to invoke it with:

AWS_PROFILE=localstack yarn serverless invoke --stage local --function hello

I get another error:

Using custom endpoint for Lambda: http://127.0.0.1:4566
{
    "errorMessage": "Unable to find executor for Lambda function \"bref-test-local-hello\". Note that Node.js, Golang, and .Net Core Lambdas currently require LAMBDA_EXECUTOR=docker",
    "errorType": "ClientError",
    "stackTrace": [
        "  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 1423, in do_execute\n    execute_result = lambda_function_callable(inv_context.event, context)\n",
        "  File \"/opt/code/localstack/localstack/services/awslambda/lambda_api.py\", line 700, in generic_handler\n    raise ClientError(\n"
    ]
}
Environment: darwin, node 16.13.1, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2
Credentials: Local, "localstack" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Invoked function failed
error Command failed with exit code 1.

@ramsey
Copy link

ramsey commented Feb 25, 2023

I did a bit more fiddling around and I think I'm at the point where it's working properly now. 😄

I found that I need to set HOSTNAME_EXTERNAL when running localstack start, so this is what I did:

HOSTNAME_EXTERNAL=127.0.0.1 localstack start -d

Now, when I deploy to localstack, it deploys properly:

AWS_PROFILE=localstack yarn serverless deploy --stage local
✔ Service deployed to stack bref-test-local (12s)

functions:
  hello: bref-test-local-hello (1.1 MB)

And when I invoke the function, it at least appears to be properly calling Lambda and getting a valid response, even if it is an error:

Using custom endpoint for Lambda: http://127.0.0.1:4566
{
    "errorMessage": "RequestId: 95efa042-d86d-128f-d35a-8c5fccbd03d1 Error: Couldn't find valid bootstrap(s): [/var/task/bootstrap /opt/bootstrap]",
    "errorType": "exitError"
}
Environment: darwin, node 16.13.1, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2
Credentials: Local, "localstack" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Invoked function failed
error Command failed with exit code 1.

I think this is because it can't load the PHP layers from Bref. So, now I think I've come to a limitation of the tool. If I'm using anything that uses layers (e.g. https://bref.sh), are those only supported in Localstack Pro?

@joe4dev
Copy link
Member

joe4dev commented Feb 27, 2023

Hi @ramsey Thank you for sharing your experience with LocalStack and PHP/Bref.

I think this is because it can't load the PHP layers from Bref. So, now I think I've come to a limitation of the tool. If I'm using anything that uses layers (e.g. https://bref.sh/), are those only supported in Localstack Pro?

Shared layers and mounting Lambda layers are a LocalStack Pro feature (see Lambda Layers).

PHP/Bref Sample

We are happy to share our new lambda-php-bref-cdk-app pro-sample (PR#207) supporting both typed PHP Lambda handlers  and traditional php-fpm applications (e.g., Laravel). Please pull the latest docker image docker pull localstack/localstack-pro and start LocalStack with the configurations PROVIDER_OVERRIDE_LAMBDA=v2 (new lambda provider implementation) and LAMBDA_DOCKER_FLAGS=--user nobody (workaround for php-fpm; see issue#7722).

Other Issues

  • serverless-localstack v1.0.4 uses an IPv4 fallback but manual configuration of host: http://127.0.0.1 takes precedence. Maybe CloudFormation fails to download the public AWS layer because that is a Pro feature.
  • "Unable to find executor for Lambda function \"bref-test-local-hello\". Note that Node.js, Golang, and .Net Core Lambdas currently require LAMBDA_EXECUTOR=docker"
    • Lambda and layers require access to Docker. Hence, please mount the Docker socket via -v "/var/run/docker.sock:/var/run/docker.sock" such that LocalStack running in Docker can launch Lambda functions using Docker.
    • We recommend using the docker-compose-pro.yml with pre-configured volume mounts and port mappings.

@Kalanamith
Copy link

Kalanamith commented Feb 27, 2023

image: localstack/localstack:0.13 works for me
Before building the localsack make sure to remove any terraform .lock or .state files (if using terraform)

@joe4dev
Copy link
Member

joe4dev commented Feb 27, 2023

image: localstack/localstack:0.13 works for me

In combination with serverless-localstack 1.0.4?

We recommend updating to the latest version of LocalStack because v0.13 is quite outdated and not supported anymore.

@Kalanamith
Copy link

@joe4dev the latest version does not work it gives the error Port 4566 not accessible

@joe4dev
Copy link
Member

joe4dev commented Feb 28, 2023

Hi @Kalanamith

  1. How do you install and start LocalStack? Remember to pull the latest Docker image.

  2. Can you reach the health endpoint curl localhost:4566/_localstack/health?

  3. Can you share the error message using serverless-localstack 1.0.4?

@joe4dev
Copy link
Member

joe4dev commented May 17, 2023

This issue occurred on macOS with Nodejs >= v17 but is now fixed #210 and also works with AWS variables after fixing the timing of getParameter SSM #219. The plugin dynamically reconfigures the LocalStack endpoint to 127.0.0.1 (IPv4) if localhost is not reachable (through IPv6). Full explanation here.

👉👉👉 Upgrade to "serverless-localstack": "^1.1.0"


For older versions, manually forcing IPv4 works as confirmed by @omerls-pw

custom:
  localstack:
    stages:
      - local
    host: http://127.0.0.1

fixed it.

🙏

Closing this issue as resolved. Please let us know if the problem persists.

@joe4dev joe4dev closed this as completed May 17, 2023
@firedancellc
Copy link

still happening on github actions localstack-pro

@joe4dev
Copy link
Member

joe4dev commented Aug 21, 2023

still happening on github actions localstack-pro

Hi @firedancellc

Can you please create a separate issue and provide some more details:

  • How do you start LocalStack?
  • What version of serverless-localstack and LocalStack are you using?
  • Is LocalStack up and running properly? Check via curl http://localhost:4566/_localstack/health
  • When does the issue happen?
  • What did you try to mitigate the issue?
  • Preferably, provide a minimal example to reproduce the issue.

That would help a lot to reproduce the issue 🙏

@Dmtreaqq
Copy link

Dmtreaqq commented Jul 15, 2024

I was using not serverless-localstack but just docker + localstack, but had same issue

Changed 'localhost' to '127.0.0.1' and it works fine. But thing is, on another mac everything was fine

@joe4dev
Copy link
Member

joe4dev commented Jul 15, 2024

I was using not serverless-localstack but just docker + localstack, but had same issue

Changed 'localhost' to '127.0.0.1' and it works fine. But thing is, on another mac everything was fine

@Dmtreaqq Were you using Node.js v17 or later?
The behavior of the breaking Node.js change depends on the DNS configuration of the machine, which could explain the different behavior on different machines.
Full explanation in localstack/aws-cdk-local#76 (comment).

@Dmtreaqq
Copy link

Dmtreaqq commented Jul 16, 2024

@joe4dev both machines Node.js 20 and both Macs but different models.

Fun fact. Something happened with my network settings today that I couldn't go into internet (any WIFI hotspot) anymore. Maybe it's totally unrelated, but I needed to delete a list of files from SystemConfiguration. And NOW I changed back to localhost:4566 and it is working.

Path: /Library/Preferences/SystemConfiguration/
Files: preferences.plist, NetworkInterfaces.plist, com.apple.airport.preferences.plist, com.apple.network.eapolclient.configuration.plist, com.apple.wifi.message-tracer.plist

Context: exernal Node.js backend app connects (send commands) to DynamoDB inside localstack with docker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests