forked from weewx/weewx
-
Notifications
You must be signed in to change notification settings - Fork 0
/
DEV_NOTES.txt
381 lines (292 loc) · 12.5 KB
/
DEV_NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
checklist for doing a release:
1. Check weewx.conf for local changes. In particular, check to make sure:
1. WEEWX_ROOT points to /home/weewx
2. debug=0 in weewx.conf
2. Make sure the version is correct
1. modify bin/weewx/__init__.py
2. make version
3. Make sure all changes have been logged
1. docs/changes.txt
2. make deb-changelog
3. make redhat-changelog
4. make suse-changelog
5. docs/upgrading.htm
4. Create the packages
1. make src-package
2. make debian-packages
3. make redhat-packages
4. make suse-packages
5. Sanity check on production machine
1. unpack the tarball, install, run
6. Tag it
1. git tag -a v1.3.1 -m "Version 1.3.1 release"
7. Upload to weewx.com
1. make upload-src
2. make upload-debian
3. make upload-redhat
4. make upload-suse
6. make upload-docs
8. Update the deb repository
1. make pull-apt-repo
2. make update-apt-repo
3. make push-apt-repo
9. Update the redhat repository
1. make pull-yum-repo
2. make update-yum-repo
3. make push-yum-repo
10. Update the suse repository
1. make pull-suse-repo
2. make update-suse-repo
3. make push-suse-repo
11. Reshuffle files on the server
1. make release
2. make release-apt-repo
3. make release-yum-repo
4. make release-suse-repo
12. Announce the release to the weewx user's group.
pre-requisites ----------------------------------------------------------------
Signing packages requires gpg and local copy of authorized private+public keys.
The debian repo management requires aptly.
The redhat repo management requires createrepo.
The suse repo management requires createrepo.
To build debian package install the following (use 'apt-get install'):
git
rsync
gpg
debhelper
python3-distutils
To build redhat package install the following (use 'yum install'):
git
rsync
rpm-build
rpm-sign
To build suse package install the following (use 'zypper install'):
git
rsync
rpm-build
howto -------------------------------------------------------------------------
how to update the version number:
perl -pi -e 's/__version__=".*"/__version__="x.y.z"/' bin/weewx/__init__.py
make version # this propagates the version number everywhere
git commit -a -m "bump to version x.y.z"
how to build source package:
make src-package
how to build debian packages:
make deb-changelog
emacs pkg/debian/changelog # add any package-specific changes, if any
git commit -m "update deb changelog" pkg/debian/changelog
make debian-packages
how to build redhat packages:
make redhat-changelog
emacs pkg/changelog.el # add any package-specific changes, if any
git commit -m "update redhat changelog" pkg/changelog.el
make redhat-packages
how to build redhat packages:
make suse-changelog
emacs pkg/changelog.suse # add any package-specific changes, if any
git commit -m "update suse changelog" pkg/changelog.suse
make suse-packages
to build redhat packages with custom rpm revision:
make redhat-changelog RPMREVISION=2
make redhat-packages RPMREVISION=2
make pull-yum-repo
make update-yum-repo RPMREVISION=2
make push-yum-repo
ssh weewx.com rsync -arv /var/www/html/yum-test/ /var/www/html/yum
suse also uses RPMREVISION, but debian uses DEBREVISION. this is useful when
there is a change to asset(s) in the packaging, but not part of weewx itself.
to display debconf variables:
sudo debconf-show weewx
to manually purge debconf variables:
echo PURGE | sudo debconf-communicate weewx
to sign rpm packages you need .rpmmacros in your home directory:
~/.rpmmacros
%_gpg_name YOUR_NAME_HERE
to generate gpg key used for signing packages:
gpg --gen-key
gpg --list-keys
to export the text version of a public key:
gpg --export -a "YOUR_NAME_HERE" > username.gpg.key
source install:
python ./setup.py install
python ./setup.py install home=/opt/weewx-x.y.z (this does not work!)
debian install/remove:
apt-get install weewx # install with apt
apt-get remove weewx # remove with apt
apt-get purge weewx # purge removes /etc/weewx and debconf settings
dpkg -i weewx_x.y.z-r.deb # install
(apt-get install weewx) # finish install if dependencies failed
dpkg -r weewx # remove
dpkg -P weewx # purge
redhat install/remove:
yum install weewx-x.y.z-r.rpm [--nogpgcheck] # install with yum
yum remove weewx # remove with yum
rpm -i weewx-x.y.z-r.rpm # install with rpm directly
rpm -e weewx # remove with rpm
suse install/remove:
zypper install weewx-x.y.z-r.rpm [--nogpgcheck] # install with zypper
zypper remove weewx # remove with zypper
rpm -i weewx-x.y.z-r.rpm # install with rpm directly
rpm -e weewx # remove with rpm
howto: deb repository ---------------------------------------------------------
aptly has two different mechanisms for doing a 'publish': switch or update.
we use snapshots, and publish using 'publish switch', rather than publishing
using a simple 'publish update'.
there are two apt repositories: python2 and python3
to do apt repo updates you must first install aptly:
https://www.aptly.info/download/
for example, on debian:
echo "deb http://repo.aptly.info/ squeeze main" | sudo tee /etc/apt/sources.list.d/aptly.list
wget -qO - https://www.aptly.info/pubkey.txt | sudo apt-key add -
sudo apt-get update
sudo apt-get install aptly
create local debian repo using aptly:
aptly repo create -distribution=squeeze -component=main -architectures=all python2-weewx
aptly repo create -distribution=buster -component=main -architectures=all python3-weewx
put a bunch of deb files into an empty apt repo:
for f in `ls distdir`; do aptly repo add python2-weewx distdir/$f; done
create a snapshot:
aptly snapshot create python-weewx-x.y.z-n from repo python2-weewx
aptly snapshot create python3-weewx-x.y.z-n from repo python3-weewx
publish using snapshot:
aptly publish -architectures=all snapshot python-weewx-x.y.z-n python2
aptly publish -architectures=all snapshot python3-weewx-x.y.z-n python3
update using 'publish switch':
aptly repo add python2-weewx dist/python-weewx_x.y.z-n_all.deb
aptly snapshot create python-weewx-x.y.z-n from repo python2-weewx
aptly publish switch squeeze python2 python-weewx-x.y.z-n
aptly repo add python3-weewx dist/python3-weewx_x.y.z-n_all.deb
aptly snapshot create python3-weewx-x.y.z-n from repo python3-weewx
aptly publish switch buster python3 python3-weewx-x.y.z-n
update using 'publish update':
aptly publish repo -architectures=all python2-weewx squeeze
aptly repo add python2-weewx dist/squeeze/python-weewx_x.y.z-n_all.deb
aptly publish update squeeze python2
aptly publish repo -architectures=all python3-weewx buster
aptly repo add python3-weewx dist/buster/python3-weewx_x.y.z-n_all.deb
aptly publish update buster python3
clone the published apt repo to local space:
mkdir -p ~/.aptly
rsync -arv [email protected]:/var/www/html/aptly-test/ ~/.aptly
synchronize local aptly changes with the published apt repo:
rsync -arv ~/.aptly/ [email protected]:/var/www/html/aptly-test
switch from testing to production (this is done at weewx.com):
rsync -arv /var/www/html/aptly-test/ /var/www/html/aptly
for clients to use an apt repo at weewx.com:
curl -s http://weewx.com/keys.html | sudo apt-key add -
echo "deb [arch=all] http://weewx.com/apt/ squeeze main" | sudo tee /etc/apt/sources.list.d/weewx.list
echo "deb [arch=all] http://weewx.com/apt/ buster main" | sudo tee /etc/apt/sources.list.d/python3-weewx.list
howto: yum repository ---------------------------------------------------------
create yum repo:
mkdir -p ~/.yum/weewx/{el7,el8}/RPMS
update local yum repo with latest rpm:
cp *.el7.rpm ~/.yum/weewx/el7/RPMS
createrepo -o ~/.yum/weewx/el7 ~/.yum/weewx/el7
cp *.el8.rpm ~/yum/weewx/el8/RPMS
createrepo -o ~/.yum/weewx/el8 ~/.yum/weewx/el8
clone the published yum repo to local space:
mkdir -p ~/.yum
rsync -arv [email protected]:/var/www/html/yum-test/ ~/.yum
synchronize local yum changes with published yum repo:
rsync -arv ~/.yum/ [email protected]:/var/www/html/yum-test
switch from testing to production (this is done at weewx.com):
rsync -arv /var/www/html/yum-test/ /var/www/html/yum
notes -------------------------------------------------------------------------
there are multiple changelogs:
docs/changes.txt - definitive changelog for the application
pkg/debian/changelog - changes to the debian packaging
pkg/changelog.el - changes to the redhat packaging
pkg/changelog.suse - changes to the suse packaging
when signing, gpg info must match the name and email in the latest package
changelog entry.
the debian changelog *must* have a version number that matches the app version.
the redhat package will build if the version numbers do not match. use the
rpm-changelog and deb-changelog targets to ensure that changelog versions match
the application version for a release.
there are many ways to build a debian package. first tried dpkg (uses DEBIAN
dir and is fairly low-level) but that does not create changes and source diffs.
then dried dpkg-buildpackage (uses debian dir and is higher level) but misses
the config and templates. ended up using dpkg-buildpackage with some manual
(scripted) file manipulation.
both debian and redhat deprecate the use of '/usr/bin/env python' as the
shebang - they want a very tight binding to the operating system python as the
shebang. in fact, since late 2019 redhat build tools see any shebang other
than a tight coupling to the operating system's python as an error and refuse
to accept it. however, the bsd platforms prefer this as the shebang, and the
other approach fails on bsd. macos uses something else.
weewx supports python 2.7 and any python 3.5+ so it should not be tightly
coupled to any specific python installation.
so use the source code should use the '/usr/bin/env python' shebang. on
platforms that refuse to accept this, replace the package builder will replace
this with whatever that platform will accept. the /etc/default/weewx plus
shell stubs in /usr/bin/wee* is used in platform installations to provide
python flexibility so that users do not have to install weewx multiple times
to experiment with different versions of python. users who run weewx directly
from a git clone get all of the 'env python' goodness of being able to specify
whatever python interpreter they want.
unit tests --------------------------------------------------------------------
prerequisites:
python 2.7
python-usb
python-mock 0.6
pyephem
to set up mysql server with user and permissions for testing:
make test-setup
to run all unit tests:
make test
(note: do not run this as root)
to clean up after running tests:
make test-clean
guidelines:
unit tests should put transient files in /var/tmp/weewx_test
testing -----------------------------------------------------------------------
what to test when creating debian and redhat packages:
install, upgrade, remove, purge
install, modify files, remove
install, modify files, upgrade
any linux:
- new install using setup.py to /home/weewx
setup.py install
- upgrade using setup.py to /home/weewx
setup.py install
modify /home/weewx/weewx.conf
setup.py install
- new install using setup.py to /opt/weewx
setup.py install home=/opt/weewx
- upgrade using setup.py to /opt/weewx
setup.py install home=/opt/weewx
modify /opt/weewx/weewx.conf
setup.py install home=/opt/weewx
on centos and suse:
- new install using rpm
rpm -i weewx_x.y.z.rpm
- upgrade using rpm
rpm -i weewx_r.s.t.rpm
rpm -U weewx_x.y.z.rpm
- upgrade using rpm with extensions installed
rpm -i weewx_r.s.t.rpm
wee_extension --install cmon
rpm -U weewx_x.y.z.rpm
debian:
- new install usinb dpkg
dpkg -i weewx_x.y.z.deb
- upgrade using dpkg take maintainer's version of weewx.conf
dpkg -i weewx_r.s.t.deb
modify /etc/weewx/weewx.conf
dpkg -i weewx_x.y.z.deb
- upgrade using dpkg use old version of weewx.conf
dpkg -i weewx_r.s.t.deb
modify /etc/weewx/weewx.conf
dpkg -i weewx_x.y.z.deb
- reconfigure using dpkg
dpkg-reconfigure weewx
all platforms:
- installation and removal of extensions
wee_extension --install cmon
wee_extension --install pmon
wee_extension --remove cmon
- reconfigure using wee_config
wee_config --reconfigure
wee_config --reconfigure --driver=weewx.drivers.vantage
- list drivers
wee_config --list-drivers