Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

DKAN Integration Branch #26

Open
wants to merge 135 commits into
base: 7.x-1.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
76fb970
Move all the work done on USDA to the upstream repo
rhabbachi Apr 20, 2016
e34f655
Document post_execute harvest date handler methode
rhabbachi Apr 21, 2016
f565e7e
Add "Datasets count" to the harvest dashboard
rhabbachi Apr 21, 2016
07e821f
Remove unneeded diff file
rhabbachi Apr 22, 2016
fd07ecc
Rename POD => datajson to match ckan naming
rhabbachi Apr 22, 2016
eec5774
Fix type hinting for string in php
rhabbachi Apr 26, 2016
67d1911
Deregister harvest if the source node is deleted
rhabbachi Apr 26, 2016
b471ad5
Fix bug getting the migration last update
rhabbachi Apr 26, 2016
934d87d
Drop and replace old harvest drush commands
rhabbachi Apr 27, 2016
673e8ed
Add disable_rules flag to harvest migration
rhabbachi Apr 27, 2016
91c1668
Added dkanextension to composer.json and created harvest.feature file.
achraf-jeday Apr 27, 2016
2088f5b
Small harvest base class and command tweaks
rhabbachi Apr 29, 2016
ec311d2
More work on the behat tests
rhabbachi May 3, 2016
c2528d5
Set the harvest object id as the uuid identifier
rhabbachi May 10, 2016
a9121fa
Remove redundant field_contact_email field mapping
rhabbachi May 10, 2016
314993f
Better resource link allowed extensions check
rhabbachi May 10, 2016
73d8776
Remove depercated dkan_harvest_example module
rhabbachi May 13, 2016
86826d0
Move helper classes to "include" directory
rhabbachi May 13, 2016
2c680e4
Merge dkan_harvest_sources into the base module
rhabbachi May 13, 2016
5afecf3
Rename DataJson tests and update phpunit.xml
rhabbachi May 13, 2016
152ed23
More work on merging dkan_harvest_sources
rhabbachi May 13, 2016
11beea7
Better dkan_harvest feature name
rhabbachi May 16, 2016
66f282c
Drop rules disabling drush option in favor of drupal variable
rhabbachi May 19, 2016
9429721
Add skeleton unit tests from usda
rhabbachi May 19, 2016
daf345e
Add harvest source types caching
rhabbachi May 26, 2016
e3fa528
Use update callback name in the unit test
rhabbachi May 26, 2016
a11d1ae
Add harvest source dataset list view
rhabbachi Jun 7, 2016
b1d2bd5
Add path_breadcrumbs export to harvest source
rhabbachi Jun 7, 2016
f8a77ae
Harvest Source re-export from vanilla DKAN
rhabbachi Jun 7, 2016
925d779
Add per-harvest source log table with items status
rhabbachi Jun 13, 2016
29c96f3
Fix datajson cacheing issuing different cache files
rhabbachi Jun 13, 2016
c7618d9
Fix constructor initialization in HarvestCache
rhabbachi Jun 15, 2016
ca68266
Refactor and add tests for harvest source log table
rhabbachi Jun 15, 2016
cfc108f
Add circleci integration for phpunit tests
rhabbachi Jun 15, 2016
93ced52
Drop travis ci integration
rhabbachi Jun 15, 2016
8884bdb
Move composer.json to make circleci happy
rhabbachi Jun 15, 2016
123110a
Update dkan-module-init-phpunit-only.sh with corret module name
rhabbachi Jun 15, 2016
58f6eeb
Fix circleci integration configuration/scripts
rhabbachi Jun 15, 2016
12529ea
Drop assertion against drupal_rmdir
rhabbachi Jun 15, 2016
2fa06a3
Add harvest source log table tests for the inital run
rhabbachi Jun 16, 2016
7b99d4a
Separate phpunit and behat tests into their own dir
rhabbachi Jun 16, 2016
5b29426
Drop travis build status from README
rhabbachi Jun 16, 2016
b58e2cc
Change missing test declaration from missing to incomplete
rhabbachi Jun 16, 2016
cc07a60
Fix the PHP notice caused by methode overrides
rhabbachi Jun 17, 2016
f5969c1
Add event page tab to the harvest source page
rhabbachi Jun 17, 2016
2fd2b27
Add TableSort to the harvest event log
rhabbachi Jun 20, 2016
7ac20c1
Add zombie entries clean-up step in postImport
rhabbachi Jun 22, 2016
e94ca4f
Add psudo node moderation and status phpunit tests
rhabbachi Jun 22, 2016
635fd62
Fix harvest source data path
rhabbachi Jun 22, 2016
8fe9e1f
Add phpunit test for zombie harvest map entries
rhabbachi Jun 22, 2016
9674b7c
Fix issues during hook_node_delete
rhabbachi Jun 23, 2016
0df0f39
Add specific class to the harvest event log table
rhabbachi Jun 24, 2016
e45cd78
Fix E_NOTICE php errors
rhabbachi Jun 24, 2016
7da27dc
Revert commit 9674b7c
rhabbachi Jun 24, 2016
106dbdf
Support field_harvest_source_ref in harvest migration
rhabbachi Jun 28, 2016
24fe5e1
Write some Documentation
rhabbachi Jul 4, 2016
8bf076d
Set map overriding warning default to false
rhabbachi Jul 7, 2016
b174e0e
Small code clean-up for the datajson import
rhabbachi Jul 7, 2016
67df0c6
Initial Harvest Source Type README documentation
rhabbachi Jul 7, 2016
b10e0f5
Formatting
rhabbachi Jul 7, 2016
d0708d8
Docs for resources and metadata imports
rhabbachi Jul 7, 2016
ef03c72
Add missing return to overriden addFiledMapping()
rhabbachi Jul 7, 2016
1f669c8
Added new formater for harvest source ref to show information about t…
jacintocapote Jul 7, 2016
84c26b6
Removed field harvest source ref definition and move to module dkan h…
jacintocapote Jul 8, 2016
0026b89
Check is set sourceid on harvest item to show info into dataset in an…
jacintocapote Jul 8, 2016
438fd0b
Remove warning message with harvest information on datasets and updat…
jacintocapote Jul 11, 2016
30bf0fb
Added two new fields (harvest source uri, last harvest performed) to …
jacintocapote Jul 11, 2016
23c2f46
Fix problem with source field on the top for harvest information into…
jacintocapote Jul 12, 2016
1b1e215
Fix problem with visualization field source harvest on dataset pages …
jacintocapote Jul 12, 2016
437c8f5
Removed formatter and field harvest ref and moved to dkan_dataset #CI…
jacintocapote Jul 12, 2016
d981c34
Modified field help texts on karvest source form.
Jul 19, 2016
35e3aef
Updated dkan branch.
Jul 19, 2016
5b04239
Fixed module init script to support dkan branch param.
Jul 19, 2016
304c6a1
Fixed module init script to support dkan branch param.
Jul 19, 2016
c4d7f42
Added import of resources in DatajsonHarvestMigration class.
Jul 21, 2016
1db0ddd
Merge pull request #21 from NuCivic/3343-new-help-text
topicus Jul 21, 2016
4c19c3c
Fixed: Resources are reimported only if dataset changed.
Jul 22, 2016
ff94add
Fixed phpunit tests.
Jul 22, 2016
3979b25
+ Brings CI setup up to date with 7.x-1.x
teosibileau Jul 21, 2016
241d990
Ups php memory
teosibileau Jul 22, 2016
46f3924
Removes dkan tests (only run dkan_harvest behat tests)
teosibileau Jul 22, 2016
e63357e
Setting dkan to harvest_dkan_integration
teosibileau Jul 22, 2016
5a991b7
Updating n of search results found
teosibileau Jul 22, 2016
8575eff
Merge pull request #24 from NuCivic/harvest_dkan_integration-fixes_ci…
MSolv Jul 22, 2016
1463ad7
Moved resource creation to complete() and removed field_resource mapp…
Jul 22, 2016
a750fe5
Merge branch 'harvest_dkan_integration' of github.com:NuCivic/dkan_ha…
Jul 22, 2016
8a938e9
Code cleanup.
Jul 22, 2016
4f9a336
Merge pull request #25 from NuCivic/3455-add-resource-support
topicus Jul 22, 2016
cdd0daa
Added warning message on edit form dataset and resource if is from a …
jacintocapote Jul 26, 2016
6bda61b
Merge pull request #27 from NuCivic/warn_message_harvest_civic_3188
dafeder Jul 27, 2016
c785320
Changed link in source field to harvest node instead url source #CIVI…
jacintocapote Jul 28, 2016
3783a14
Merge branch '7.x-1.x' into harvest_dkan_integration
topicus Aug 1, 2016
5daecfe
Merge pull request #28 from NuCivic/information_dataset_harvest_sourc…
dafeder Aug 2, 2016
9853bcb
Added tab preview to node to show info before import with harvest #CI…
jacintocapote Jul 19, 2016
ff1784e
Added test for check preview tab on harvest integration #CIVIC-3339
jacintocapote Jul 20, 2016
3dde377
Added redirect to preview page after save a harvest resouce. #CIVIC-2695
jacintocapote Jul 27, 2016
1ce0aeb
Redirect to preview tab after create a harvest resource and add butto…
jacintocapote Jul 27, 2016
f30e27e
Added description to harvest content type
jacintocapote Jul 27, 2016
710fa6b
Added description to harvest preview table #CIVIC-2695
jacintocapote Jul 27, 2016
f941954
Removed file dkan_harvest.pages.inc and moved code to .module #CIVIC-…
jacintocapote Jul 27, 2016
65faa7b
Disable node previews for sourcee, remove harvest source ref field
dafeder Aug 2, 2016
f338b04
Merge pull request #22 from NuCivic/harvest_preview_civic_2695
dafeder Aug 2, 2016
c605013
Adding view for harvested datasets (#20)
janette Aug 5, 2016
5b1821d
Update wording stuff and add no results messages
topicus Aug 9, 2016
876c169
Reset and rollback idle migrations
topicus Aug 9, 2016
5745703
Fix timefloor declaration
topicus Aug 9, 2016
aede3b0
Add error tab test
topicus Aug 9, 2016
46d19ea
Add dkan_harvest dependency to dkan_harvest_dashboard (#35)
dafeder Aug 9, 2016
d6b5ced
Show help text even though there is no results
topicus Aug 9, 2016
bf22dfc
Fixes to dkan_harvest features to avoid overridden state (#36)
dafeder Aug 9, 2016
197713f
Merge pull request #34 from NuCivic/error_handling
dafeder Aug 9, 2016
7b5fd05
Documentation updates (#37)
dafeder Aug 9, 2016
a62496e
Add group import (#39)
MSolv Aug 12, 2016
398e780
Fix tab order
topicus Aug 18, 2016
e4f6b80
Add field orphan to resources. Fix filters, overrides, defaults and e…
topicus Aug 19, 2016
fdaec8b
Merge branch 'harvest_dkan_integration' into civic-3884-remediate-bugs
topicus Aug 19, 2016
9667c89
Add display improvements to harvest source summary block
janette Aug 26, 2016
03ebf5f
Use theme function to create summary block
janette Aug 26, 2016
f08910b
Fix undefined index
janette Aug 26, 2016
d0e3dcb
civic-3890 update labels on exposed filters
janette Aug 26, 2016
20a3047
Merge pull request #42 from NuCivic/civic-3884-remediate-bugs
dafeder Aug 29, 2016
d42baed
Remove 'groups were updated...' messages when harvesting datasets
janette Aug 29, 2016
0572ff0
Fix node-view dataset count function
janette Aug 31, 2016
808a25e
Merge pull request #45 from NuCivic/fix-count
dafeder Sep 1, 2016
eaf627d
Add more specific regex check on the messages
janette Sep 1, 2016
11b12f4
Merge pull request #44 from NuCivic/civic-3886
dafeder Sep 2, 2016
5a695b8
Modified confirmation screen for delete VBO on sources dashboard.
Oct 1, 2016
8d17301
Hidden harvest related fields on dataset and resources forms.
Oct 1, 2016
bcb60b2
Fixed warning on add resource form.
Oct 1, 2016
2d7e4ab
Refactored delete confirmation screen to be used on non VBO delete.
Oct 1, 2016
d4b0628
Added finished callback on source delete batch process.
Oct 3, 2016
260e28c
Added check.
Oct 3, 2016
fb677b4
Deleted tests forlder.
Oct 6, 2016
eb050a5
Removed copy of features from circle ci conf file.
Oct 6, 2016
3f9c257
Pull latest updates from DKAN core
rhabbachi Dec 20, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .probo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ steps:
export DATABASE_URL="mysql://root:[email protected]/dkan"
time bash dkan-module-init.sh --deps --build=$DATABASE_URL
#Set a symlink so that probo can serve the site.

echo ""
echo "-------> Symlink ./docroot to var/www/html:"
echo ""
Expand All @@ -41,4 +40,4 @@ steps:
echo "Known Issues (1/22/2016):"
echo " - Site may give a SQL error on initial load. Give it a few seconds and then refresh and it should work. This is due to the container apache waking up before the MYSQL. Probo.ci knows, but they say it's not at the top of the priority since few are having that issue."
echo " - You should usually choose: the Build Permalink (blue button) for doing site QA. If you don't any new commits may blow away the site while you are viewing it as well as the data. 'View Site' Green button shows the latest build, so it can change while you are interacting with it."
echo ""
echo ""
28 changes: 0 additions & 28 deletions .travis.yml

This file was deleted.

367 changes: 322 additions & 45 deletions README.md

Large diffs are not rendered by default.

59 changes: 0 additions & 59 deletions circle.yml

This file was deleted.

10 changes: 0 additions & 10 deletions composer.json

This file was deleted.

62 changes: 0 additions & 62 deletions dkan-module-init.sh

This file was deleted.

27 changes: 13 additions & 14 deletions dkan_harvest.api.inc
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,21 @@

/**
* @file
* dkan_harvest.api.inc.
* Documentation for hooks defined by Dkan Harvest.
*/

function dkan_harvest_harvest_sources() {
/**
* Register a Dkan Harvest Source type.
*
* @see HarvestSourceType
*/
function dkan_harvest_harvest_sources_types() {
return array(
'source_id' => array(
'remote' => 'http://data_json_remote',
'type' => 'data.json',
// Filter items preseting the following values (Optional).
'filters' => array('keyword' => array('health')),
// Exclude items presenting the following values (Optional).
'excludes' => array('keyword' => array('tabacco')),
// Provide defaults (Optional).
'defaults' => array('keyword' => array('harvested dataset'),
// Provide overrides (Optional).
'overrides' => array('author' => 'Author'),
'harvest_type_json' => array(
'machine_name' => 'harvest_type_json',
'name' => 'Harvest Source Type JSON',
'cache callback' => 'mymodule_harvest_type_json',
'migration class' => 'TypeHarvestMigrate',
),
);
}
}
Loading