Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Schema Violation errors during extraction #3

Open
dsprayberry opened this issue Sep 14, 2020 · 4 comments
Open

Schema Violation errors during extraction #3

dsprayberry opened this issue Sep 14, 2020 · 4 comments

Comments

@dsprayberry
Copy link

A Zoom integration running tap-zoom version 0.0.1 encountered schema violations during transformation, seemingly from the report_webinars stream based on the included fields, though the affected stream is unclear from the exception. Selected streams for this connection include:

  • report_webinar_participants
  • report_webinars
  • users
  • webinar_absentees
  • webinar_files
  • webinar_panelists
  • webinar_poll_results
  • webinar_polls
  • webinar_qna_results
  • webinar_questions
  • webinar_registrants
  • webinar_tracking_sources
  • webinars

Full traceback:

2020-09-14 17:01:04,910Z tap - Traceback (most recent call last):
2020-09-14 17:01:04,911Z tap - File "tap-env/bin/tap-zoom", line 33, in
2020-09-14 17:01:04,911Z tap - sys.exit(load_entry_point('tap-zoom==0.0.1', 'console_scripts', 'tap-zoom')())
2020-09-14 17:01:04,911Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 229, in wrapped
2020-09-14 17:01:04,911Z tap - return fnc(*args, **kwargs)
2020-09-14 17:01:04,911Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/init.py", line 41, in main
2020-09-14 17:01:04,911Z tap - parsed_args.state)
2020-09-14 17:01:04,911Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 138, in sync
2020-09-14 17:01:04,911Z tap - {})
2020-09-14 17:01:04,911Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 89, in sync_endpoint
2020-09-14 17:01:04,911Z tap - child_key_bag)
2020-09-14 17:01:04,911Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 89, in sync_endpoint
2020-09-14 17:01:04,911Z tap - child_key_bag)
2020-09-14 17:01:04,911Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 72, in sync_endpoint
2020-09-14 17:01:04,911Z tap - mdata)
2020-09-14 17:01:04,912Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/transform.py", line 140, in transform
2020-09-14 17:01:04,912Z tap - raise SchemaMismatch(self.errors)
2020-09-14 17:01:04,912Z tap - singer.transform.SchemaMismatch: Errors during transform
2020-09-14 17:01:04,912Z tap - dept: data does not match {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}
2020-09-14 17:01:04,912Z tap - : data does not match {'type': 'object', 'selected': True, 'additionalProperties': False, 'properties': {'id': {'type': ['integer'], 'selected': True, 'inclusion': 'available'}, 'user_email': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'dept': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'tracking_fields': {'type': ['array', 'null'], 'selected': True, 'items': {'type': ['object'], 'additionalProperties': False, 'properties': {'value': {'type': ['null', 'string']}, 'field': {'type': ['null', 'string']}}}, 'inclusion': 'available'}, 'webinar_id': {'type': ['string'], 'selected': True, 'inclusion': 'available'}, 'duration': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'total_minutes': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'type': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'start_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'topic': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'end_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'user_name': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'uuid': {'type': ['string'], 'selected': True, 'inclusion': 'automatic'}, 'participants_count': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}}}
2020-09-14 17:01:04,912Z tap -
2020-09-14 17:01:04,912Z tap -
2020-09-14 17:01:04,912Z tap - Errors during transform: [dept: data does not match {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, : data does not match {'type': 'object', 'selected': True, 'additionalProperties': False, 'properties': {'id': {'type': ['integer'], 'selected': True, 'inclusion': 'available'}, 'user_email': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'dept': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'tracking_fields': {'type': ['array', 'null'], 'selected': True, 'items': {'type': ['object'], 'additionalProperties': False, 'properties': {'value': {'type': ['null', 'string']}, 'field': {'type': ['null', 'string']}}}, 'inclusion': 'available'}, 'webinar_id': {'type': ['string'], 'selected': True, 'inclusion': 'available'}, 'duration': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'total_minutes': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'type': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'start_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'topic': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'end_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'user_name': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'uuid': {'type': ['string'], 'selected': True, 'inclusion': 'automatic'}, 'participants_count': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}}}]

Full extraction logs in case they're helpful:

2020-09-14 17:00:49,418Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2020-09-14 17:00:49,423Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2020-09-14 17:00:50,433Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [32] at offset None
2020-09-14 17:00:50,434Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-zoom --config /tmp/tap_discover_config.json --discover
2020-09-14 17:00:50,752Z    tap - INFO Testing authentication
2020-09-14 17:00:50,812Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.05941438674926758, tags={'http_status_code': 200, 'status': 'succeeded'})
2020-09-14 17:00:50,813Z    tap - INFO Starting discover
2020-09-14 17:00:50,828Z    tap - INFO Finished discover
2020-09-14 17:00:50,885Z   main - INFO Tap exited normally.
2020-09-14 17:00:50,887Z   main - INFO Saving list of discovered streams
2020-09-14 17:00:54,008Z   main - INFO Saving structure of stream report_meeting_participants (tap_stream_id: report_meeting_participants)
2020-09-14 17:00:55,302Z   main - INFO Saving structure of stream webinar_polls (tap_stream_id: webinar_polls)
2020-09-14 17:00:55,353Z   main - INFO Saving structure of stream webinar_qna_results (tap_stream_id: webinar_qna_results)
2020-09-14 17:00:55,865Z   main - INFO Saving structure of stream webinar_tracking_sources (tap_stream_id: webinar_tracking_sources)
2020-09-14 17:00:55,937Z   main - INFO Saving structure of stream report_meetings (tap_stream_id: report_meetings)
2020-09-14 17:00:55,985Z   main - INFO Saving structure of stream meeting_questions (tap_stream_id: meeting_questions)
2020-09-14 17:00:56,025Z   main - INFO Saving structure of stream users (tap_stream_id: users)
2020-09-14 17:00:56,081Z   main - INFO Saving structure of stream webinar_questions (tap_stream_id: webinar_questions)
2020-09-14 17:00:56,125Z   main - INFO Saving structure of stream webinar_registrants (tap_stream_id: webinar_registrants)
2020-09-14 17:00:56,173Z   main - INFO Saving structure of stream meeting_files (tap_stream_id: meeting_files)
2020-09-14 17:00:56,248Z   main - INFO Saving structure of stream webinar_absentees (tap_stream_id: webinar_absentees)
2020-09-14 17:00:56,312Z   main - INFO Saving structure of stream webinar_poll_results (tap_stream_id: webinar_poll_results)
2020-09-14 17:00:56,368Z   main - INFO Saving structure of stream meeting_poll_results (tap_stream_id: meeting_poll_results)
2020-09-14 17:00:56,431Z   main - INFO Saving structure of stream report_webinar_participants (tap_stream_id: report_webinar_participants)
2020-09-14 17:00:56,570Z   main - INFO Saving structure of stream meeting_registrants (tap_stream_id: meeting_registrants)
2020-09-14 17:00:56,626Z   main - INFO Saving structure of stream report_webinars (tap_stream_id: report_webinars)
2020-09-14 17:00:56,731Z   main - INFO Saving structure of stream meetings (tap_stream_id: meetings)
2020-09-14 17:00:58,691Z   main - INFO Saving structure of stream webinar_panelists (tap_stream_id: webinar_panelists)
2020-09-14 17:00:58,812Z   main - INFO Saving structure of stream webinars (tap_stream_id: webinars)
2020-09-14 17:00:59,011Z   main - INFO Saving structure of stream meeting_polls (tap_stream_id: meeting_polls)
2020-09-14 17:00:59,072Z   main - INFO Saving structure of stream webinar_files (tap_stream_id: webinar_files)
2020-09-14 17:00:59,129Z   main - INFO Writing catalog to file
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinars: id, agenda, timezone, created_at, occurences, start_url, duration, tracking_fields, start_time, settings, uuid, recurrence, join_url, topic, type, host_id
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_poll_results: email, webinar_uuid, question_details, name
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_absentees: address, zip, org, state, comments, status, county, job_title, no_of_employees, create_time, industry, id, first_name, purchasing_time_frame, join_url, city, phone, custom_questions, email, webinar_uuid, role_in_purchase_process, last_name
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_polls: id, questions, webinar_id, status, title
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream report_webinars: id, participants_count, user_email, user_name, topic, duration, tracking_fields, start_time, uuid, dept, webinar_id, end_time, type, total_minutes
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_questions: custom_questions, questions, webinar_id
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream users: id, pmi, timezone, first_name, dept, im_group_ids, last_login_time, created_at, last_client_version, status, email, type, group_ids, last_name, verified
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_qna_results: email, webinar_uuid, question_details, name
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_registrants: address, zip, state, comments, status, county, job_title, no_of_employees, create_time, industry, id, first_name, purchasing_time_frame, join_url, city, webinar_id, phone, custom_questions, email, org, role_in_purchase_process, last_name
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_files: download_url, file_size, webinar_uuid, file_name
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_panelists: id, email, join_url, webinar_id, name
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream webinar_tracking_sources: id, webinar_id, source_name, tracking_url, visitor_count, registration_count
2020-09-14 17:00:59,153Z   main - INFO Selected fields for stream report_webinar_participants: id, attentiveness_score, webinar_id, leave_time, user_email, join_time, duration, name, user_id
2020-09-14 17:00:59,258Z   main - INFO Current state: {"currently_syncing": "users"}
2020-09-14 17:00:59,259Z   main - INFO Starting tap: tap-env/bin/tap-zoom --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2020-09-14 17:00:59,263Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: meeting_files
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: meeting_poll_results
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: meeting_registrants
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: meetings
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: meeting_polls
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: report_meeting_participants
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: report_meetings
2020-09-14 17:00:59,616Z    tap - INFO Skipping stream: meeting_questions
2020-09-14 17:00:59,710Z    tap - INFO HTTP request to "users" endpoint took 0.092s, returned status code 200
2020-09-14 17:00:59,738Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.024764299392700195, tags={'endpoint': 'list_webinars', 'status': 'succeeded'})
2020-09-14 17:00:59,758Z target - INFO Using batch_size_prefernces of {'user_batch_size_preference': None, 'full_table_streams': [], 'batch_size_preference': None}
2020-09-14 17:00:59,758Z   main - INFO State update: adding currently_syncing = "users"
2020-09-14 17:00:59,764Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.024234294891357422, tags={'endpoint': 'list_webinars', 'status': 'succeeded'})
2020-09-14 17:00:59,793Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.02692556381225586, tags={'endpoint': 'list_webinars', 'status': 'succeeded'})
2020-09-14 17:00:59,823Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.02739691734313965, tags={'endpoint': 'list_webinars', 'status': 'succeeded'})
2020-09-14 17:00:59,859Z    tap - INFO HTTP request to "list_webinars" endpoint took 0.034s, returned status code 200
2020-09-14 17:00:59,860Z target - INFO Serializing batch with 5 messages for table users
2020-09-14 17:00:59,861Z target - INFO Sending batch of 4045 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:00:59,888Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.027252912521362305, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:00:59,938Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.049s, returned status code 200
2020-09-14 17:00:59,940Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:00:59,940Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:00:59,940Z target - INFO Sending batch of 2040 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:00:59,978Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.037s, returned status code 200
2020-09-14 17:00:59,980Z    tap - INFO replicated 4 records from "webinar_panelists" endpoint
2020-09-14 17:00:59,980Z target - INFO Serializing batch with 4 messages for table webinar_panelists
2020-09-14 17:00:59,980Z target - INFO Sending batch of 1946 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,010Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.029s, returned status code 200
2020-09-14 17:01:00,011Z    tap - INFO replicated 0 records from "webinar_polls" endpoint
2020-09-14 17:01:00,066Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.054s, returned status code 200
2020-09-14 17:01:00,068Z    tap - INFO replicated 3 records from "report_webinar_participants" endpoint
2020-09-14 17:01:00,068Z target - INFO Serializing batch with 3 messages for table report_webinar_participants
2020-09-14 17:01:00,068Z target - INFO Sending batch of 2078 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,104Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.035s, returned status code 200
2020-09-14 17:01:00,106Z    tap - INFO replicated 0 records from "webinar_registrants" endpoint
2020-09-14 17:01:00,136Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.030s, returned status code 200
2020-09-14 17:01:00,138Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:00,198Z    tap - INFO HTTP request to "webinars" endpoint took 0.058s, returned status code 200
2020-09-14 17:01:00,200Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:00,201Z target - INFO Sending batch of 7257 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,236Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.034540414810180664, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:00,278Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04098081588745117, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:00,319Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03930187225341797, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:00,356Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03463482856750488, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:00,356Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:00,397Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04013514518737793, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:00,460Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.061s, returned status code 200
2020-09-14 17:01:00,461Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:01:00,461Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:01:00,462Z target - INFO Sending batch of 2041 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,501Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.039s, returned status code 200
2020-09-14 17:01:00,503Z    tap - INFO replicated 5 records from "webinar_panelists" endpoint
2020-09-14 17:01:00,503Z target - INFO Serializing batch with 5 messages for table webinar_panelists
2020-09-14 17:01:00,504Z target - INFO Sending batch of 2248 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,536Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.031s, returned status code 200
2020-09-14 17:01:00,537Z    tap - INFO replicated 0 records from "webinar_polls" endpoint
2020-09-14 17:01:00,612Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.074s, returned status code 200
2020-09-14 17:01:00,614Z    tap - INFO replicated 2 records from "report_webinar_participants" endpoint
2020-09-14 17:01:00,615Z target - INFO Serializing batch with 2 messages for table report_webinar_participants
2020-09-14 17:01:00,615Z target - INFO Sending batch of 1724 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,650Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.035s, returned status code 200
2020-09-14 17:01:00,651Z    tap - INFO replicated 0 records from "webinar_registrants" endpoint
2020-09-14 17:01:00,694Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.042s, returned status code 200
2020-09-14 17:01:00,695Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:00,802Z    tap - INFO HTTP request to "webinars" endpoint took 0.105s, returned status code 200
2020-09-14 17:01:00,804Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:00,805Z target - INFO Sending batch of 6865 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:00,842Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03666114807128906, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:00,876Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03324532508850098, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:00,911Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03400540351867676, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:00,944Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03163313865661621, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:00,944Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:00,972Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.027176856994628906, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:01,029Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.055s, returned status code 200
2020-09-14 17:01:01,030Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:01:01,031Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:01:01,031Z target - INFO Sending batch of 2035 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,066Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.034s, returned status code 200
2020-09-14 17:01:01,068Z    tap - INFO replicated 3 records from "webinar_panelists" endpoint
2020-09-14 17:01:01,068Z target - INFO Serializing batch with 3 messages for table webinar_panelists
2020-09-14 17:01:01,068Z target - INFO Sending batch of 1437 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,102Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.031s, returned status code 200
2020-09-14 17:01:01,102Z    tap - INFO replicated 0 records from "webinar_polls" endpoint
2020-09-14 17:01:01,256Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.153s, returned status code 200
2020-09-14 17:01:01,259Z    tap - INFO replicated 3 records from "report_webinar_participants" endpoint
2020-09-14 17:01:01,260Z target - INFO Serializing batch with 3 messages for table report_webinar_participants
2020-09-14 17:01:01,260Z target - INFO Sending batch of 2075 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,293Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.031s, returned status code 200
2020-09-14 17:01:01,294Z    tap - INFO replicated 0 records from "webinar_registrants" endpoint
2020-09-14 17:01:01,326Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.032s, returned status code 200
2020-09-14 17:01:01,327Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:01,429Z    tap - INFO HTTP request to "webinars" endpoint took 0.100s, returned status code 200
2020-09-14 17:01:01,431Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:01,431Z target - INFO Sending batch of 7185 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,466Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03422379493713379, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:01,505Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03745007514953613, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:01,547Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04071784019470215, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:01,581Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03206801414489746, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:01,581Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:01,612Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.02992415428161621, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:01,667Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.054s, returned status code 200
2020-09-14 17:01:01,670Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:01:01,670Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:01:01,670Z target - INFO Sending batch of 2045 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,714Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.043s, returned status code 200
2020-09-14 17:01:01,715Z    tap - INFO replicated 5 records from "webinar_panelists" endpoint
2020-09-14 17:01:01,715Z target - INFO Serializing batch with 5 messages for table webinar_panelists
2020-09-14 17:01:01,716Z target - INFO Sending batch of 2241 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,751Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.035s, returned status code 200
2020-09-14 17:01:01,752Z    tap - INFO replicated 0 records from "webinar_polls" endpoint
2020-09-14 17:01:01,825Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.072s, returned status code 200
2020-09-14 17:01:01,831Z    tap - INFO replicated 7 records from "report_webinar_participants" endpoint
2020-09-14 17:01:01,832Z target - INFO Serializing batch with 7 messages for table report_webinar_participants
2020-09-14 17:01:01,832Z target - INFO Sending batch of 3525 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:01,875Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.042s, returned status code 200
2020-09-14 17:01:01,876Z    tap - INFO replicated 0 records from "webinar_registrants" endpoint
2020-09-14 17:01:01,910Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.032s, returned status code 200
2020-09-14 17:01:01,911Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:01,998Z    tap - INFO HTTP request to "webinars" endpoint took 0.084s, returned status code 200
2020-09-14 17:01:02,001Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:02,002Z target - INFO Sending batch of 6984 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:02,048Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04633522033691406, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:02,092Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03837466239929199, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:02,131Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04166722297668457, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:02,191Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.05846238136291504, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:02,192Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:02,319Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.12621212005615234, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:02,379Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:01:02,380Z target - INFO Sending batch of 2020 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:02,380Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.053s, returned status code 200
2020-09-14 17:01:02,388Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:01:02,425Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.042s, returned status code 200
2020-09-14 17:01:02,427Z    tap - INFO replicated 9 records from "webinar_panelists" endpoint
2020-09-14 17:01:02,429Z target - INFO Serializing batch with 9 messages for table webinar_panelists
2020-09-14 17:01:02,430Z target - INFO Sending batch of 3132 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:02,472Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.036s, returned status code 200
2020-09-14 17:01:02,473Z    tap - INFO replicated 0 records from "webinar_polls" endpoint
2020-09-14 17:01:02,589Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.115s, returned status code 200
2020-09-14 17:01:02,636Z    tap - INFO replicated 73 records from "report_webinar_participants" endpoint
2020-09-14 17:01:02,637Z target - INFO Serializing batch with 73 messages for table report_webinar_participants
2020-09-14 17:01:02,637Z target - INFO Sending batch of 27355 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:02,678Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.039s, returned status code 200
2020-09-14 17:01:02,681Z    tap - INFO replicated 3 records from "webinar_registrants" endpoint
2020-09-14 17:01:02,682Z target - INFO Serializing batch with 3 messages for table webinar_registrants
2020-09-14 17:01:02,682Z target - INFO Sending batch of 4418 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:02,721Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.037s, returned status code 200
2020-09-14 17:01:02,722Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:02,810Z    tap - INFO HTTP request to "webinars" endpoint took 0.086s, returned status code 200
2020-09-14 17:01:02,817Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:02,817Z target - INFO Sending batch of 7089 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:02,857Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03782916069030762, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:02,897Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.037835121154785156, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:02,935Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03632235527038574, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:02,971Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.033721923828125, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:02,971Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:03,002Z    tap - INFO HTTP request to "webinar_questions" endpoint took 0.028s, returned status code 200
2020-09-14 17:01:03,004Z    tap - INFO replicated 1 records from "webinar_questions" endpoint
2020-09-14 17:01:03,004Z target - INFO Serializing batch with 1 messages for table webinar_questions
2020-09-14 17:01:03,004Z target - INFO Sending batch of 1001 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,038Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03314614295959473, tags={'endpoint': 'report_webinars', 'status': 'succeeded'})
2020-09-14 17:01:03,070Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.030s, returned status code 200
2020-09-14 17:01:03,071Z    tap - INFO replicated 0 records from "webinar_panelists" endpoint
2020-09-14 17:01:03,112Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.040s, returned status code 200
2020-09-14 17:01:03,114Z    tap - INFO replicated 0 records from "webinar_polls" endpoint
2020-09-14 17:01:03,151Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03556036949157715, tags={'endpoint': 'report_webinar_participants', 'status': 'succeeded'})
2020-09-14 17:01:03,198Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.044s, returned status code 200
2020-09-14 17:01:03,199Z    tap - INFO replicated 0 records from "webinar_registrants" endpoint
2020-09-14 17:01:03,246Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.045s, returned status code 200
2020-09-14 17:01:03,248Z    tap - INFO replicated 1 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:03,251Z target - INFO Serializing batch with 1 messages for table webinar_tracking_sources
2020-09-14 17:01:03,251Z target - INFO Sending batch of 980 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,309Z    tap - INFO HTTP request to "webinars" endpoint took 0.057s, returned status code 200
2020-09-14 17:01:03,311Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:03,312Z target - INFO Sending batch of 5654 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,350Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.038100481033325195, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:03,392Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.040392398834228516, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:03,426Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.032204627990722656, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:03,460Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.0328984260559082, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:03,461Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:03,502Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03985595703125, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:03,616Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.112s, returned status code 200
2020-09-14 17:01:03,618Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:01:03,618Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:01:03,619Z target - INFO Sending batch of 2016 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,654Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.034s, returned status code 200
2020-09-14 17:01:03,655Z    tap - INFO replicated 3 records from "webinar_panelists" endpoint
2020-09-14 17:01:03,656Z target - INFO Serializing batch with 3 messages for table webinar_panelists
2020-09-14 17:01:03,656Z target - INFO Sending batch of 1437 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,702Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.044s, returned status code 200
2020-09-14 17:01:03,704Z    tap - INFO replicated 1 records from "webinar_polls" endpoint
2020-09-14 17:01:03,706Z target - INFO Serializing batch with 1 messages for table webinar_polls
2020-09-14 17:01:03,706Z target - INFO Sending batch of 1133 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,790Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.083s, returned status code 200
2020-09-14 17:01:03,794Z    tap - INFO replicated 2 records from "report_webinar_participants" endpoint
2020-09-14 17:01:03,795Z target - INFO Serializing batch with 2 messages for table report_webinar_participants
2020-09-14 17:01:03,795Z target - INFO Sending batch of 1704 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,840Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.043s, returned status code 200
2020-09-14 17:01:03,841Z    tap - INFO replicated 1 records from "webinar_registrants" endpoint
2020-09-14 17:01:03,841Z target - INFO Serializing batch with 1 messages for table webinar_registrants
2020-09-14 17:01:03,842Z target - INFO Sending batch of 2742 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,875Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.033s, returned status code 200
2020-09-14 17:01:03,876Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:03,938Z    tap - INFO HTTP request to "webinars" endpoint took 0.059s, returned status code 200
2020-09-14 17:01:03,942Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:03,942Z target - INFO Sending batch of 5862 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:03,986Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04186844825744629, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:04,016Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.028987884521484375, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:04,051Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03362083435058594, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:04,088Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03531694412231445, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:04,088Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:04,118Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.029132843017578125, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:04,204Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.085s, returned status code 200
2020-09-14 17:01:04,206Z    tap - INFO replicated 1 records from "report_webinars" endpoint
2020-09-14 17:01:04,206Z target - INFO Serializing batch with 1 messages for table report_webinars
2020-09-14 17:01:04,206Z target - INFO Sending batch of 2015 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:04,249Z    tap - INFO HTTP request to "webinar_panelists" endpoint took 0.041s, returned status code 200
2020-09-14 17:01:04,250Z    tap - INFO replicated 4 records from "webinar_panelists" endpoint
2020-09-14 17:01:04,251Z target - INFO Serializing batch with 4 messages for table webinar_panelists
2020-09-14 17:01:04,251Z target - INFO Sending batch of 1727 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:04,358Z    tap - INFO HTTP request to "webinar_polls" endpoint took 0.104s, returned status code 200
2020-09-14 17:01:04,359Z    tap - INFO replicated 1 records from "webinar_polls" endpoint
2020-09-14 17:01:04,360Z target - INFO Serializing batch with 1 messages for table webinar_polls
2020-09-14 17:01:04,360Z target - INFO Sending batch of 1097 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:04,459Z    tap - INFO HTTP request to "report_webinar_participants" endpoint took 0.098s, returned status code 200
2020-09-14 17:01:04,466Z    tap - INFO replicated 17 records from "report_webinar_participants" endpoint
2020-09-14 17:01:04,467Z target - INFO Serializing batch with 17 messages for table report_webinar_participants
2020-09-14 17:01:04,468Z target - INFO Sending batch of 7110 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:04,504Z    tap - INFO HTTP request to "webinar_registrants" endpoint took 0.036s, returned status code 200
2020-09-14 17:01:04,507Z    tap - INFO replicated 3 records from "webinar_registrants" endpoint
2020-09-14 17:01:04,508Z target - INFO Serializing batch with 3 messages for table webinar_registrants
2020-09-14 17:01:04,508Z target - INFO Sending batch of 3728 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:04,546Z    tap - INFO HTTP request to "webinar_tracking_sources" endpoint took 0.035s, returned status code 200
2020-09-14 17:01:04,547Z    tap - INFO replicated 0 records from "webinar_tracking_sources" endpoint
2020-09-14 17:01:04,623Z    tap - INFO HTTP request to "webinars" endpoint took 0.074s, returned status code 200
2020-09-14 17:01:04,627Z target - INFO Serializing batch with 1 messages for table webinars
2020-09-14 17:01:04,627Z target - INFO Sending batch of 5910 bytes to https://api.stitchdata.com/v2/import/batch
2020-09-14 17:01:04,664Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.036644935607910156, tags={'endpoint': 'webinar_absentees', 'status': 'succeeded'})
2020-09-14 17:01:04,725Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.059514522552490234, tags={'endpoint': 'webinar_poll_results', 'status': 'succeeded'})
2020-09-14 17:01:04,775Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.04863119125366211, tags={'endpoint': 'webinar_qna_results', 'status': 'succeeded'})
2020-09-14 17:01:04,811Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03459978103637695, tags={'endpoint': 'webinar_files', 'status': 'succeeded'})
2020-09-14 17:01:04,811Z    tap - INFO replicated 1 records from "webinars" endpoint
2020-09-14 17:01:04,846Z    tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.03328680992126465, tags={'endpoint': 'webinar_questions', 'status': 'succeeded'})
2020-09-14 17:01:04,905Z    tap - INFO HTTP request to "report_webinars" endpoint took 0.058s, returned status code 200
2020-09-14 17:01:04,907Z    tap - INFO replicated 0 records from "report_webinars" endpoint
2020-09-14 17:01:04,908Z    tap - INFO replicated 0 records from "list_webinars" endpoint
2020-09-14 17:01:04,908Z    tap - INFO replicated 5 records from "users" endpoint
2020-09-14 17:01:04,909Z    tap - CRITICAL Errors during transform
2020-09-14 17:01:04,910Z    tap - 	dept: data does not match {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}
2020-09-14 17:01:04,910Z    tap - 	: data does not match {'type': 'object', 'selected': True, 'additionalProperties': False, 'properties': {'id': {'type': ['integer'], 'selected': True, 'inclusion': 'available'}, 'user_email': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'dept': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'tracking_fields': {'type': ['array', 'null'], 'selected': True, 'items': {'type': ['object'], 'additionalProperties': False, 'properties': {'value': {'type': ['null', 'string']}, 'field': {'type': ['null', 'string']}}}, 'inclusion': 'available'}, 'webinar_id': {'type': ['string'], 'selected': True, 'inclusion': 'available'}, 'duration': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'total_minutes': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'type': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'start_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'topic': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'end_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'user_name': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'uuid': {'type': ['string'], 'selected': True, 'inclusion': 'automatic'}, 'participants_count': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}}}
2020-09-14 17:01:04,910Z    tap - 
2020-09-14 17:01:04,910Z    tap - 
2020-09-14 17:01:04,910Z    tap - Errors during transform: [dept: data does not match {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, : data does not match {'type': 'object', 'selected': True, 'additionalProperties': False, 'properties': {'id': {'type': ['integer'], 'selected': True, 'inclusion': 'available'}, 'user_email': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'dept': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'tracking_fields': {'type': ['array', 'null'], 'selected': True, 'items': {'type': ['object'], 'additionalProperties': False, 'properties': {'value': {'type': ['null', 'string']}, 'field': {'type': ['null', 'string']}}}, 'inclusion': 'available'}, 'webinar_id': {'type': ['string'], 'selected': True, 'inclusion': 'available'}, 'duration': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'total_minutes': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'type': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'start_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'topic': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'end_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'user_name': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'uuid': {'type': ['string'], 'selected': True, 'inclusion': 'automatic'}, 'participants_count': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}}}]
2020-09-14 17:01:04,910Z    tap - Traceback (most recent call last):
2020-09-14 17:01:04,911Z    tap -   File "tap-env/bin/tap-zoom", line 33, in <module>
2020-09-14 17:01:04,911Z    tap -     sys.exit(load_entry_point('tap-zoom==0.0.1', 'console_scripts', 'tap-zoom')())
2020-09-14 17:01:04,911Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 229, in wrapped
2020-09-14 17:01:04,911Z    tap -     return fnc(*args, **kwargs)
2020-09-14 17:01:04,911Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/__init__.py", line 41, in main
2020-09-14 17:01:04,911Z    tap -     parsed_args.state)
2020-09-14 17:01:04,911Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 138, in sync
2020-09-14 17:01:04,911Z    tap -     {})
2020-09-14 17:01:04,911Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 89, in sync_endpoint
2020-09-14 17:01:04,911Z    tap -     child_key_bag)
2020-09-14 17:01:04,911Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 89, in sync_endpoint
2020-09-14 17:01:04,911Z    tap -     child_key_bag)
2020-09-14 17:01:04,911Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_zoom/sync.py", line 72, in sync_endpoint
2020-09-14 17:01:04,911Z    tap -     mdata)
2020-09-14 17:01:04,912Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/transform.py", line 140, in transform
2020-09-14 17:01:04,912Z    tap -     raise SchemaMismatch(self.errors)
2020-09-14 17:01:04,912Z    tap - singer.transform.SchemaMismatch: Errors during transform
2020-09-14 17:01:04,912Z    tap - 	dept: data does not match {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}
2020-09-14 17:01:04,912Z    tap - 	: data does not match {'type': 'object', 'selected': True, 'additionalProperties': False, 'properties': {'id': {'type': ['integer'], 'selected': True, 'inclusion': 'available'}, 'user_email': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'dept': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'tracking_fields': {'type': ['array', 'null'], 'selected': True, 'items': {'type': ['object'], 'additionalProperties': False, 'properties': {'value': {'type': ['null', 'string']}, 'field': {'type': ['null', 'string']}}}, 'inclusion': 'available'}, 'webinar_id': {'type': ['string'], 'selected': True, 'inclusion': 'available'}, 'duration': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'total_minutes': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'type': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'start_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'topic': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'end_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'user_name': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'uuid': {'type': ['string'], 'selected': True, 'inclusion': 'automatic'}, 'participants_count': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}}}
2020-09-14 17:01:04,912Z    tap - 
2020-09-14 17:01:04,912Z    tap - 
2020-09-14 17:01:04,912Z    tap - Errors during transform: [dept: data does not match {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, : data does not match {'type': 'object', 'selected': True, 'additionalProperties': False, 'properties': {'id': {'type': ['integer'], 'selected': True, 'inclusion': 'available'}, 'user_email': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'dept': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'tracking_fields': {'type': ['array', 'null'], 'selected': True, 'items': {'type': ['object'], 'additionalProperties': False, 'properties': {'value': {'type': ['null', 'string']}, 'field': {'type': ['null', 'string']}}}, 'inclusion': 'available'}, 'webinar_id': {'type': ['string'], 'selected': True, 'inclusion': 'available'}, 'duration': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'total_minutes': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'type': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}, 'start_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'topic': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'end_time': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available', 'format': 'date-time'}, 'user_name': {'type': ['string', 'null'], 'selected': True, 'inclusion': 'available'}, 'uuid': {'type': ['string'], 'selected': True, 'inclusion': 'automatic'}, 'participants_count': {'type': ['integer', 'null'], 'selected': True, 'inclusion': 'available'}}}]
2020-09-14 17:01:04,975Z target - INFO Requests complete, stopping loop
2020-09-14 17:01:05,077Z   main - INFO Target exited normally with status 0
2020-09-14 17:01:05,437Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [33] at offset None
2020-09-14 17:01:05,438Z   main - INFO No tunnel subprocess to tear down
2020-09-14 17:01:05,438Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "Errors during transform". Target succeeded.```
@kekoav kekoav mentioned this issue Sep 17, 2020
Merged
@kekoav
Copy link
Contributor

kekoav commented Sep 17, 2020

@dylan-stitch @cmerrick I have a fix in #4 , I tested the tap locally.

I did notice there were additional fields from the API not in the schema, I added a couple I saw here, but this should probably be audited for field additions to Zoom's API.

@adamluba1
Copy link

I am experiencing the same error. Are there any plans to merge the fix?

@jdotjdot
Copy link

jdotjdot commented Oct 29, 2020 •
edited
Loading

This is still an issue, even if you choose not to track the users table. The integration simply doesn't work at the moment.

lfpll pushed a commit to lfpll/tap-zoom that referenced this issue Dec 13, 2022
@mryorik
[DE-2391] Prepare local development environment for Composer (singer-…
268c2b5 
…io#3)
@rdeshmukh15
Copy link
Contributor

@kekoav @jdotjdot @adamluba1: Is this issue still persists?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@kekoav @jdotjdot @dsprayberry @adamluba1 @rdeshmukh15