Commit fb88dd37 authored by Dimitri Podborski's avatar Dimitri Podborski
Browse files

Merge branch 'cli' into 'master'

Cli

Closes #191

See merge request !4
parents 36e3030b 44b29008
......@@ -39,16 +39,25 @@ deactivate
## 2. systems.py
`systems.py` is a script which is intended to be used in the Systems group. Use the `-h` option to see all available parameters. This script allows you to:
`systems.py` is a script which is intended to be used in the Systems group. Use the `-h` option to see all available parameters.
To provide input data to the script you can either:
- use `--csv` option to use the CSV file
- or use `-m` (`--documents`) together with `-p` (`--project`) options
1. Print information about input documents on MDMS and GitLab (`-p` is optional):
e.g.: `python systems.py -m m55958,m55959,m56121 -p FileFormat/CENC --meeting 133`
2. Open issues based on the information provided in a CSV file:
e.g.: `python systems.py --csv Contribs.csv -o`
Below are a few examples:
1. Open issues based on the information provided in a CSV file:
e.g.: `python systems.py -o --csv Contribs.csv`
2. Open issues based on CLI options:
e.g.: `python systems.py -o -m 55958,55959,56121 -p FileFormat/CENC --meeting 133`
3. Generate an output document based on the information provided in a CSV file. Use a template as a basis (`--template` is optional):
e.g.: `python systems.py --csv Contribs.csv -d --template templates/WG03_input_template_dimitri.docx`
4. Close issues basd on the information provided in a CSV file:
e.g.: `python systems.py --csv Contribs.csv -c`
e.g.: `python systems.py -d --csv Contribs.csv --template template.docx`
4. Close issues based on the information provided in a CSV file:
e.g.: `python systems.py -c --csv Contribs.csv`
5. Close issues based on CLI options:
e.g.: `python systems.py -c -m m55958,m55959,m56121 -p FileFormat/CENC --meeting 133`
6. Print information about input documents on MDMS and GitLab:
e.g.: `python systems.py -l -m m55958,m55959,m56121 -p FileFormat/CENC --meeting 133`
The CSV file must have a header row with the folowing entries:
......@@ -56,8 +65,8 @@ The CSV file must have a header row with the folowing entries:
- To determine which GitLab project needs to be used make sure that your CSV file has either:
- **Project URL** - a full URL to your GitLab project (*recommended*)
- or **Sub Group** and **Project Name** - two last elements of the Project URL. (non-case-sensitive)
e.g.: http://mpegx.int-evry.fr/software/MPEG/Systems/PCC-SYS/V-PCC → `Sub Group = PCC-SYS` and `Project Name = V-PCC`.
e.g.: http://mpegx.int-evry.fr/software/MPEG/Systems/FileFormat/isobmff → `Sub Group = FileFormat` and `Project Name = ISOBMFF`
e.g.: http://mpegx.int-evry.fr/software/MPEG/Systems/PCC-SYS/V-PCC → `Sub Group=PCC-SYS`, `Project Name=V-PCC`.
e.g.: http://mpegx.int-evry.fr/software/MPEG/Systems/FileFormat/isobmff → `Sub Group=FileFormat`, `Project Name=ISOBMFF`
- **Close issue** - is required if you want to close multiple issues at once. Supported values are `0`, `1`, `TRUE`, `FALSE`, `true` and `false`.
The example CSV below has both `Project URL` and (`Sub Group` with `Project Name`) but you can also have one of these in your CSV. The CSV delimiter is determined automatically.
......
......@@ -4,6 +4,7 @@ Some helper functions
'''
import json
import os
import re
from datetime import datetime, timedelta
from docx import Document, opc, oxml, shared
from docx.enum.text import WD_ALIGN_PARAGRAPH # pylint: disable=E0611
......@@ -163,7 +164,7 @@ def find_project(projects, url_or_path, path_root = 'MPEG/'):
'''
if url_or_path is None:
return None
striped_url_or_path = url_or_path.replace(' ', '').strip()
striped_url_or_path = url_or_path.replace(' ', '').strip().strip('/')
for project in projects:
if striped_url_or_path == project['url']:
return project
......@@ -213,7 +214,7 @@ def create_issue_metadata(document, details):
if len(details['documents']) > 0:
last_doc = max(details['documents'], key=lambda x:x['version'])
version = str(last_doc['version'])
title = document['title'].replace('(', '').replace(')', '')
title = document['title'].replace('(', '').replace(')', '').replace(',', '')
meta = '[meta]: # ({},{},{},{})'.format(document['mdms_id'], document['document'], title, version)
return meta
......@@ -245,7 +246,7 @@ def create_issue_description(document, details):
description += details['abstract']
else:
description += '* [ ] please **add your abstract here**.\n'
description += '* [ ] please also **add your abstract to MDMS** (this will be used when we create the output document).\n'
description += '* [ ] please also **add your abstract to MDMS** (this can be used when we create the output document).\n'
description += '\n\n_automatically generated issue_'
return description
......@@ -258,4 +259,22 @@ def get_updated_issue_description(current_decription, document, details):
return None
description = create_issue_description_header(document, details)
description += current_decription[pos1+len(CLOSING_TAG):]
return description
\ No newline at end of file
return description
def find_gitlab_users(gitlab_users, document):
usernames = []
regex = re.compile(r'[^a-zA-Z\s]')
try:
for author in document['authors']:
author_name = author['name'].lower().strip()
author_name = regex.sub('', author_name) # remove non alphabetic chars
author_name = ' '.join( [w for w in author_name.split() if len(w)>1] ) # remove single letters
for key in gitlab_users:
gl_name = gitlab_users[key]['name'].lower().strip()
gl_name = regex.sub('', gl_name) # remove non alphabetic chars
gl_name = ' '.join( [w for w in gl_name.split() if len(w)>1] ) # remove single letters
if author_name == gl_name:
usernames.append(key)
except:
return []
return usernames
......@@ -4,13 +4,13 @@ This is the interface to MPEG Document Management System (mdms).
It requests the data from MDMS and parses the output HTML
'''
from urllib.parse import urljoin, parse_qs, urlparse
import os
import requests
import bs4
import re
from urllib.parse import urljoin, parse_qs, urlparse
from datetime import datetime
from enum import Enum, unique
import re
BASE_URL = 'https://dms.mpeg.expert/doc_end_user/'
MEETINGS_URL = urljoin(BASE_URL, 'all_meeting.php')
......@@ -443,4 +443,5 @@ def find_documents( title = '',
Find documents using the search URL.
TODO: Fire a POST request to SEARCH_URL and parse the result
'''
return None
\ No newline at end of file
return None
\ No newline at end of file
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment