Warm tip: This article is reproduced from stackoverflow.com, please click
python regex

How should I print formated text from a txt file with python?

发布于 2020-03-31 23:00:57

I got this file with this info pattern:

# Query 1: 204.60k QPS, 230.79x concurrency, ID XXXXXXXXXX at byte 19XXX9318
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: 2020-01-29 18:18:59.073995 to 18:18:59.074005
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count          7       2
# Exec time     10     2ms     1ms     1ms     1ms     1ms    12us     1ms
# Rows affecte   0       0       0       0       0       0       0       0
# Query size     7      74      37      37      37      37       0      37
# Warning coun   0       0       0       0       0       0       0       0
# String:
# Hosts        10.1.1.5 (1/50%), 10.8.0.2 (1/50%)
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms  ################################################################
#  10ms
# 100ms
#    1s
#  10s+
SHOW SESSION STATUS LIKE 'XXXXX'\G
\n break line 
repeat

I want to run a script in python to get only some information from that file. There will be multiple querys.

Currently Im trying something like this:

#!/usr/bin/python

file = open("/etc/openvpn/logs/log-2020_01_29_06_20_PM.txt", "r")
read = file.read()
removeChar = read.replace("#", "")
for item in removeChar.split("\n"):
        if "Hosts" and "Time range" in item:
                print item.strip()

The output is:

Time range: 2020-01-29 18:18:59.073995 to 18:18:59.074005
Time range: 2020-01-29 18:18:58.489162 to 18:18:59.188582
Time range: 2020-01-29 18:18:58.666020 to 18:18:58.666028

I want it to be something like this:

['Query 1, 2020-01-29 18:18:59, 10.1.1.5, 10.8.0.2, SHOW SESSION STATUS LIKE 'XXXXX'\G']
['Query 2, 2020-01-29 18:19:59, 10.1.1.5, 10.8.0.2, SHOW FROM BLA * LIKE 'BLA'\G']

Im tired of trying to find how to do this, also I'm learning python because it is a good language to learn! :)

Thanks.

Questioner
Henrique Mota
Viewed
20
Giannis Clipper 2020-01-31 23:30

You could try without regex, using string manipulation only:

data = '''# Query 1: 204.60k QPS, 230.79x concurrency, ID XXXXXXXXXX at byte 19XXX9318
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: 2020-01-29 18:18:59.073995 to 18:18:59.074005
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count          7       2
# Exec time     10     2ms     1ms     1ms     1ms     1ms    12us     1ms
# Rows affecte   0       0       0       0       0       0       0       0
# Query size     7      74      37      37      37      37       0      37
# Warning coun   0       0       0       0       0       0       0       0
# String:
# Hosts        10.1.1.5 (1/50%), 10.8.0.2 (1/50%)
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms  ################################################################
#  10ms
# 100ms
#    1s
#  10s+
SHOW SESSION STATUS LIKE 'XXXXX'\G
\n break line 
repeat
'''

data = data.split('\n')

all_results = []

result = []

for row in data:
    if row.startswith('# Query ') and not row.startswith('# Query size'):
        row = row.split(':')[0].split('# ')[1]
        result.append(row)

    elif row.startswith('# Hosts'):
        row = row.replace('# Hosts', '').replace(' ', '').split(',')
        result.append(row[0].split('(')[0])
        result.append(row[1].split('(')[0])

    elif row.startswith('# Time range:'):
        row = row.replace('# Time range:', '').split('.')[0].strip()
        result.append(row)

    elif row.startswith('SHOW') and row.endswith('\G'):
        result.append(row)
        result = ', '.join(result)
        all_results.append(result)
        result = []

print(all_results)

    # output: "Query 1, 2020-01-29 18:18:59, 10.1.1.5, 10.8.0.2, SHOW SESSION STATUS LIKE 'XXXXX'\\G"