Verify the Dangi calendar comparing with https://astro.kasi.re.kr/life/pageView/5

[hsivonen]

It would be prudent to check that ICU4X's Dangi calendar agrees with https://astro.kasi.re.kr/life/pageView/5

[fabon-f]

I wrote a script using Deno and icu npm package to check discrepancies between ICU4X and official Korean lunisolar calendar.

deno run --allow-net --allow-write fetch.ts korean_calendar.jsonl
deno run --allow-read check.ts korean_calendar.jsonl

source code

// fetch.ts
import { DOMParser } from "jsr:@b-fuze/deno-dom@0.1.52";
import { delay } from "jsr:@std/async@1.0.13/delay";

async function fetchPage(isoYear: number, isoMonth: number) {
  const isoMonthString = isoMonth.toString().padStart(2, "0");
  const response = await fetch("https://astro.kasi.re.kr/life/pageView/5", {
    method: "POST",
    headers: {
      "Content-Type": "application/x-www-form-urlencoded",
    },
    body: new URLSearchParams({
      search_year: isoYear.toString(),
      search_month: isoMonthString,
      search_dp: "1",
      search_check: "G",
      select: "1",
      yyyy: isoYear.toString(),
      mm: isoMonthString,
      date: `${isoYear}-${isoMonthString}`,
    }),
  });
  return response.text();
}

function exactlyOne<T>(array: ArrayLike<T>): T {
  if (array.length !== 1) {
    throw new Error();
  }
  return array[0];
}

function extractDatesFromHtml(html: string) {
  const doc = new DOMParser().parseFromString(html, "text/html");
  const table = exactlyOne(doc.querySelectorAll("#container table"));
  const headers = [
    ...exactlyOne(table.querySelectorAll("thead tr")).querySelectorAll("th"),
  ].map((e) => e.textContent);
  if (headers.length !== 5) {
    throw new Error();
  }
  const gregorianIndex = headers.indexOf("그레고리력");
  const lunarIndex = headers.indexOf("음력");
  if (gregorianIndex === -1 || lunarIndex === -1) {
    throw new Error();
  }
  return [...table.querySelectorAll("tbody tr")].map((e) => {
    const cells = e.querySelectorAll("th, td");
    return {
      iso: parseKoreanGregorianDate(cells[gregorianIndex].textContent),
      lunar: parseKoreanLunarDate(cells[lunarIndex].textContent),
    };
  });
}

function parseKoreanGregorianDate(
  date: string,
): [year: number, month: number, day: number] {
  const result = date.match(/(\d+)년 (\d\d)월 (\d\d)일/);
  if (!result) {
    throw new Error();
  }
  return [parseInt(result[1]), parseInt(result[2]), parseInt(result[3])];
}

function parseKoreanLunarDate(
  date: string,
): [year: number, monthCode: string, day: number] {
  const result = date.match(/(\d+)년 (윤)?(\d\d)월 (\d\d)일/);
  if (!result) {
    throw new Error();
  }
  return [
    parseInt(result[1]),
    result[2] !== undefined ? `M${result[3]}L` : `M${result[3]}`,
    parseInt(result[4]),
  ];
}

const outputFile = Deno.args[0];
if (outputFile === undefined) {
  console.error("Specify output file");
  Deno.exit(1);
}

Deno.writeTextFile(outputFile, "");

// from 1800 to 2050
for (let year = 1800; year <= 2050; year++) {
  for (let month = 1; month <= 12; month++) {
    console.error(`${year} ${month}`);
    const filteredDates = extractDatesFromHtml(await fetchPage(year, month))
      // only output a start or an end of a lunisolar month
      .filter((date) => date.lunar[2] < 3 || date.lunar[2] > 27);
    for (const date of filteredDates) {
      Deno.writeTextFile(outputFile, `${JSON.stringify(date)}\n`, {
        append: true,
      });
    }
    await delay(300);
  }
}

// check.ts
import * as icu from "npm:icu@2.0.4";
import * as v from "npm:valibot@1.1.0";

const inputFile = Deno.args[0];
if (inputFile === undefined) {
  console.error("Specify input file");
  Deno.exit(1);
}

const schema = v.strictObject({
  iso: v.tuple([v.number(), v.number(), v.number()]),
  lunar: v.tuple([v.number(), v.string(), v.number()]),
});

const koreanLunarCalendar = new icu.Calendar(icu.CalendarKind.Dangi);

for (const line of (await Deno.readTextFile(inputFile)).trim().split("\n")) {
  const { iso, lunar } = v.parse(schema, JSON.parse(line));
  const koreanDate = icu.Date.fromIsoInCalendar(
    ...iso,
    koreanLunarCalendar,
  );
  if (koreanDate.monthCode !== lunar[1] || koreanDate.dayOfMonth !== lunar[2]) {
    console.log(iso, lunar, [
      koreanDate.eraYearOrRelatedIso,
      koreanDate.monthCode,
      koreanDate.dayOfMonth,
    ]);
  }
}

The script checks years between 1800 and 2050. According to the script, there are some differences before 1912: in 1803, 1804, 1809, 1815, 1818, 1821, 1827, 1828, 1831, 1832, 1846, 1847, 1849, 1851, 1852, 1857, 1862, 1870, 1882, 1890, 1896, 1897, 1903, 1904, 1905, 1908, 1911 (based on related ISO year, not Gregorian years). There is no differences after 1912, including future dates until 2050.

raw data from the KASI site: https://gist.github.com/fabon-f/297420e4606fe8410b7f0348773e3918

[sffc]

This is great, thank you!

I wonder what happened in 1912. Prior to 1911, the Chinese calendar used by the Qing Dynasty had methods slightly different from the modern methods. I wonder if there's something similar going on with the Korean calendar.

[fabon-f]

The time zone offset of Korean time was UTC+08:27:52 before 1908, UTC+08:30 between 1908 and 1912, but apparently ICU4X uses UTC+09:00 for astronomical calculation. I believe these offset transitions are the most critical factor of discrepancies. Note that South Korea uses standard time with UTC+08:30 from 1954 to 1961, but fortunately it doesn't cause any differences in lunisolar calendar calculation.

Migration from apparent solar time to mean solar time can also cause discrepancies, but I don't know and couldn't find when mean time was introduced in Korea.

[fabon-f]

I found out that the comment I made above was probably not accurate.

After I checked an actual calendar published by Joseon, Korean Empire and Japanese colonial government in KASI site, I noticed that calendrical books until 1912 note all astronomical events in Beijing time (apparent solar time, I guess), and lunisolar calendar always matches to one published by Qing. Apparently it was only since 1913 that the official lunisolar calendar is calculated based on Korea Standard Time (GMT+9).

[Manishearth]

#7008

[robertbastian]

I think we should keep this open until we have a test against KASI data similar to the one we have for HKO data