Skip to content

Add sublocalities feature to properly categorize neighborhoods and districts within cities #1147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Copilot wants to merge 5 commits into from
Closed

Add sublocalities feature to properly categorize neighborhoods and districts within cities #1147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
144ccb2
Initial plan
Copilot Oct 14, 2025
94fcc3c
Add sublocalities table schema and import/export support
Copilot Oct 14, 2025
1a26bf4
Update all export commands to support sublocalities
Copilot Oct 14, 2025
6d8bb28
Add sublocalities identification script and comprehensive documentation
Copilot Oct 14, 2025
ba086fc
Add quick start guide for sub-localities feature
Copilot Oct 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,10 +201,13 @@ Perfect for developers, researchers, and businesses who need clean, structured g
| Countries | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| States | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Cities | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Sublocalities** 🆕 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Country+States | ✅ | NA | NA | NA | NA | NA | NA | NA | NA |
| Country+Cities | ✅ | NA | NA | NA | NA | NA | NA | NA | NA |
| Country+State+Cities/World | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | NA | NA | NA |

**New:** Sublocalities (neighborhoods, districts, areas within cities) are now available as a separate dataset with proper parent-city relationships. See [SUBLOCALITIES.md](docs/SUBLOCALITIES.md) for details.


## Demo

Expand All @@ -217,9 +220,12 @@ Total Sub Regions : 22 <br>
Total Countries : 250 <br>
Total States/Regions/Municipalities : 5,038 <br>
Total Cities/Towns/Districts : 151,024 <br>
**Total Sublocalities/Neighborhoods : 0** 🆕 <br>

Last Updated On : 13th Oct 2025

> **Note:** The sublocalities feature is newly added to help properly categorize neighborhoods, districts, and areas within cities (such as Bandra in Mumbai, Manhattan in New York). See [docs/SUBLOCALITIES.md](docs/SUBLOCALITIES.md) for details on how to contribute sub-locality data.

## Import MongoDB

How to import MongoDB database?
Expand Down
2 changes: 2 additions & 0 deletions bin/Commands/ExportCsv.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ class ExportCsv extends Command
'countries' => ['from' => '/json/countries.json', 'to' => '/csv/countries.csv'],
'states' => ['from' => '/json/states.json', 'to' => '/csv/states.csv'],
'cities' => ['from' => '/json/cities.json', 'to' => '/csv/cities.csv'],
'sublocalities' => ['from' => '/json/sublocalities.json', 'to' => '/csv/sublocalities.csv'],
'regions' => ['from' => '/json/regions.json', 'to' => '/csv/regions.csv'],
'subregions' => ['from' => '/json/subregions.json', 'to' => '/csv/subregions.csv'],
];
Expand All @@ -25,6 +26,7 @@ class ExportCsv extends Command
'countries' => ['from' => '/json/countries.json', 'place_type' => 'country'],
'states' => ['from' => '/json/states.json', 'place_type' => 'state'],
'cities' => ['from' => '/json/cities.json', 'place_type' => 'city'],
'sublocalities' => ['from' => '/json/sublocalities.json', 'place_type' => 'sublocality'],
'regions' => ['from' => '/json/regions.json', 'place_type' => 'region'],
'subregions' => ['from' => '/json/subregions.json', 'place_type' => 'subregion'],
];
Expand Down
27 changes: 27 additions & 0 deletions bin/Commands/ExportJson.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
$k = 0; // countries-states-cities && countries-states
$l = 0;
$m = 0; // countries
$n = 0; // sublocalities

$countriesArray = array();
$statesArray = array();
Expand All @@ -52,6 +53,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
$countryStateCityArray = array();
$regionsArray = array();
$subregionsArray = array();
$sublocalitiesArray = array();
$stateNamesArray = array();
$cityNamesArray = array();

Expand Down Expand Up @@ -273,11 +275,35 @@ protected function execute(InputInterface $input, OutputInterface $output): int
}
}

// Fetching All Sublocalities
$sql = "SELECT * FROM sublocalities ORDER BY name";
$result = $db->query($sql);
if ($result->num_rows > 0) {
while ($row = $result->fetch_assoc()) {
// Pushing it into Fresh Array
$sublocalitiesArray[$n]['id'] = (int)$row['id'];
$sublocalitiesArray[$n]['name'] = $row['name'];
$sublocalitiesArray[$n]['city_id'] = (int)$row['city_id'];
$sublocalitiesArray[$n]['state_id'] = (int)$row['state_id'];
$sublocalitiesArray[$n]['state_code'] = $row['state_code'];
$sublocalitiesArray[$n]['country_id'] = (int)$row['country_id'];
$sublocalitiesArray[$n]['country_code'] = $row['country_code'];
$sublocalitiesArray[$n]['latitude'] = $row['latitude'];
$sublocalitiesArray[$n]['longitude'] = $row['longitude'];
$sublocalitiesArray[$n]['native'] = $row['native'];
$sublocalitiesArray[$n]['timezone'] = $row['timezone'];
$sublocalitiesArray[$n]['translations'] = json_decode($row['translations'], true);
$sublocalitiesArray[$n]['wikiDataId'] = $row['wikiDataId'];
$n++;
}
}

$io->writeln('Total Regions Count : ' . count($regionsArray));
$io->writeln('Total Subregions Count : ' . count($subregionsArray));
$io->writeln('Total Countries Count : ' . count($countriesArray));
$io->writeln('Total States Count : ' . count($statesArray));
$io->writeln('Total Cities Count : ' . count($citiesArray));
$io->writeln('Total Sublocalities Count : ' . count($sublocalitiesArray));

// Add a Space
$io->newLine();
Expand All @@ -288,6 +314,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
'/json/countries.json' => $countriesArray,
'/json/states.json' => $statesArray,
'/json/cities.json' => $citiesArray,
'/json/sublocalities.json' => $sublocalitiesArray,
'/json/countries+states.json' => $countryStateArray,
'/json/countries+cities.json' => $countryCityArray,
'/json/countries+states+cities.json' => $countryStateCityArray
Expand Down
67 changes: 66 additions & 1 deletion bin/Commands/ExportMongoDB.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class ExportMongoDB extends Command
protected static $defaultName = 'export:mongodb';
protected static $defaultDescription = 'Export data to MongoDB format';

private const COLLECTIONS = ['regions', 'subregions', 'countries', 'states', 'cities'];
private const COLLECTIONS = ['regions', 'subregions', 'countries', 'states', 'cities', 'sublocalities'];
private Filesystem $filesystem;
private array $dataCache = [];

Expand Down Expand Up @@ -67,6 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
$this->processCountries($io, $rootDir);
$this->processStates($io, $rootDir);
$this->processCities($io, $rootDir);
$this->processSublocalities($io, $rootDir);

$io->success('MongoDB export completed successfully');
return Command::SUCCESS;
Expand Down Expand Up @@ -287,6 +288,70 @@ private function processCities(SymfonyStyle $io, string $rootDir): void
$io->info('Cities exported to MongoDB format');
}

private function processSublocalities(SymfonyStyle $io, string $rootDir): void
{
$io->section('Processing sublocalities');

// Handle case where sublocalities might not exist yet
if (!isset($this->dataCache['sublocalities']) || empty($this->dataCache['sublocalities'])) {
$io->warning('No sublocalities data found - skipping');
return;
}

$sublocalities = $this->dataCache['sublocalities'];
$processedSublocalities = [];

foreach ($sublocalities as $sublocality) {
$processedSublocality = $sublocality;

// Convert id to MongoDB _id format
$processedSublocality['_id'] = (int) $sublocality['id'];
unset($processedSublocality['id']);

// Parse JSON translations if it's a string
if (isset($processedSublocality['translations']) && is_string($processedSublocality['translations'])) {
$processedSublocality['translations'] = json_decode($processedSublocality['translations'], true);
}

// Add city reference
if (isset($processedSublocality['city_id'])) {
$processedSublocality['city'] = [
'$ref' => 'cities',
'$id' => (int) $processedSublocality['city_id']
];
}

// Add state reference
if (isset($processedSublocality['state_id'])) {
$processedSublocality['state'] = [
'$ref' => 'states',
'$id' => (int) $processedSublocality['state_id']
];
}

// Add country reference
if (isset($processedSublocality['country_id'])) {
$processedSublocality['country'] = [
'$ref' => 'countries',
'$id' => (int) $processedSublocality['country_id']
];
}

// Convert coordinates to GeoJSON format for MongoDB geospatial queries
if (isset($processedSublocality['latitude']) && isset($processedSublocality['longitude'])) {
$processedSublocality['location'] = [
'type' => 'Point',
'coordinates' => [(float) $processedSublocality['longitude'], (float) $processedSublocality['latitude']]
];
}

$processedSublocalities[] = $processedSublocality;
}

$this->saveCollection($rootDir, 'sublocalities', $processedSublocalities);
$io->info('Sublocalities exported to MongoDB format');
}

private function saveCollection(string $rootDir, string $collection, array $data): void
{
$outputFile = "$rootDir/mongodb/$collection.json";
Expand Down
25 changes: 24 additions & 1 deletion bin/Commands/ExportSqlServer.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class ExportSqlServer extends Command
protected static $defaultName = 'export:sql-server';
protected static $defaultDescription = 'Export data to SQL Server format';

private const TABLES = ['regions', 'subregions', 'countries', 'states', 'cities'];
private const TABLES = ['regions', 'subregions', 'countries', 'states', 'cities', 'sublocalities'];
private Filesystem $filesystem;

public function __construct()
Expand Down Expand Up @@ -135,6 +135,29 @@ private function generateTableSchema(string $table): string
wikiDataId NVARCHAR(255) NULL,
CONSTRAINT FK_cities_states FOREIGN KEY (state_id) REFERENCES world.states(id),
CONSTRAINT FK_cities_countries FOREIGN KEY (country_id) REFERENCES world.countries(id)
);",
'sublocalities' => "
IF OBJECT_ID('world.sublocalities', 'U') IS NOT NULL DROP TABLE world.sublocalities;
CREATE TABLE world.sublocalities (
id INT IDENTITY(1,1) PRIMARY KEY,
name NVARCHAR(255) NOT NULL,
city_id INT NOT NULL,
state_id INT NOT NULL,
state_code NVARCHAR(255) NOT NULL,
country_id INT NOT NULL,
country_code NCHAR(2) NOT NULL,
latitude DECIMAL(10,8) NOT NULL,
longitude DECIMAL(11,8) NOT NULL,
native NVARCHAR(255) NULL,
timezone NVARCHAR(255) NULL,
translations NVARCHAR(MAX),
created_at DATETIME2 NOT NULL DEFAULT '2014-01-01 12:01:01',
updated_at DATETIME2 NOT NULL DEFAULT GETDATE(),
flag BIT NOT NULL DEFAULT 1,
wikiDataId NVARCHAR(255) NULL,
CONSTRAINT FK_sublocalities_cities FOREIGN KEY (city_id) REFERENCES world.cities(id),
CONSTRAINT FK_sublocalities_states FOREIGN KEY (state_id) REFERENCES world.states(id),
CONSTRAINT FK_sublocalities_countries FOREIGN KEY (country_id) REFERENCES world.countries(id)
);"
];

Expand Down
1 change: 1 addition & 0 deletions bin/Commands/ExportXml.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ class ExportXml extends Command
'countries' => ['from' => '/json/countries.json', 'to' => '/xml/countries.xml', 'singular' => 'country'],
'states' => ['from' => '/json/states.json', 'to' => '/xml/states.xml', 'singular' => 'state'],
'cities' => ['from' => '/json/cities.json', 'to' => '/xml/cities.xml', 'singular' => 'city'],
'sublocalities' => ['from' => '/json/sublocalities.json', 'to' => '/xml/sublocalities.xml', 'singular' => 'sublocality'],
];

private Filesystem $filesystem;
Expand Down
1 change: 1 addition & 0 deletions bin/Commands/ExportYaml.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ class ExportYaml extends Command
'countries' => ['from' => '/json/countries.json', 'to' => '/yml/countries.yml', 'singular' => 'country'],
'states' => ['from' => '/json/states.json', 'to' => '/yml/states.yml', 'singular' => 'state'],
'cities' => ['from' => '/json/cities.json', 'to' => '/yml/cities.yml', 'singular' => 'city'],
'sublocalities' => ['from' => '/json/sublocalities.json', 'to' => '/yml/sublocalities.yml', 'singular' => 'sublocality'],
];

private Filesystem $filesystem;
Expand Down
Loading